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1  Introduction 


Abstract 


This  research  effort  is  concentrated  on  the  computational  neuroscience  of  early  vision.  Progress 
was  made  on  the  following  problems:  (i)  development  and  simulation  of  our  model  for  logi¬ 
cal/linear  interactions  in  visual  cortical  cells,  application  of  it  to  several  problems  in  visual  psy¬ 
chophysics,  and  inital  testing  of  physiological  predictions  from  it;  (ii)  extension  of  our  model 
of  endstopped  visual  cortical  neurons  into  motion  and  visual  area  MT;  (iii)  formal  development 
of  our  model  of  shape,  and  initial  psychophysical  experiments  inspired  from  it;  and  (iv)  formal 
development  of  a  model  for  inferring  global  curves  from  bcal  (tangent)  representations. 


1.  Introduction 

Our  second  year  of  research  support  by  AFOSR  Grant  89-0260  has  been  an  exciting  one. 
Real  progress  has  been  made  on  several  fronts,  especially  Computational  Neuroscience,  Theory, 
and  Neurophysiology.  Several  papers  have  resulted  from  this  research,  and  we  discuss  each 
project  in  turn.  Since  this  grant  is  an  interdisciplinary  one,  we  begin  with  a  project  in  which  our 
computational  modeling  has  already  inspired  physiological  experiments. 


2.  Logical/Linear  Modeling  of  Visual  Cortical  Cells 

Our  main  area  of  collaboration  continues  to  be  the  modeling  of  neurons  in  the  visual  cortex, 
but  now  with  a  concentration  on  which  non-linearities  might  arise  within  them,  what  computa¬ 
tional  role  these  non-linearities  could  play  in  functional  theories  of  vision,  and  how  they  can  be 
studied  physiologically.  Most  models  of  simple  cells  are  linear  (e.g.,  Gabor  functions,  differences 
of  Gaussians,  etc.),  and  these  are  combined  (via  rectification)  into  complex  cells.  Our  work 
began  with  a  computational  motivation — that  certain  non-linearities  are  extremely  important  for 
separating  nearby,  but  separate  curves — and  suggests  a  drastically  different  sort  of  non-linearity 
than  has  previously  been  investigated.  In  particular,  we  have  developed  a  calculus  for  the  bgical 
combination  of  bcal  evidence  from  subunits  in  which  support  is  accumulated  linearly,  but  incom¬ 
patible  evidence  enters  nonlinearly.  This  calculus  is  described  in  detail  in  the  Technical  Report  in 
Appendix  I.  It  leads  to  operators  that  appear  linear  for  one  class  of  stimuli  but  markedly  nonlin¬ 
ear  for  others  —  we  call  these  logical-linear  operators.  They  exhibit  dual  advantages:  they  are 
considerably  more  stimulus-specific  than  purely  linear  operators,  while  more  robust  to  incidental 
stimulus  variation  than  logical  operators.  The  viability  of  such  operators  as  models  of  visual  cor¬ 
tical  neurons  (e.g.  simple  cells)  has  been  examined  by  comparing  simulations  of  purely  linear  and 
logical-linear  models  to  the  responses  of  cortical  neurons.  Operators  with  properties  specialised 
for  spatial  contours  are  examined  with  stimuli  containing  vernier  offsets,  interruptions,  and  oppo¬ 
site  contrast  segments.  The  results  are  consistent  with  the  well-known  "linear”  properties  (e.g., 
sensitivity  to  s.f.  gratings)  while  exhibiting  the  nonlinear  behaviour  associated  with  high  vernier 
sensitivities  (Swindale  and  Cynader,  1989)  and  strong  suppressive  effects  for  opposite  contrast 
segments  (Hammond  and  MacKay,  1983,  1985).  These  latter  results  are  described  in  Appendix 
II. 
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3.  Modeling  of  Endstopping  and  Curvature 


Our  computational  studies  of  logical/linear  interactions  have  been  so  exciting  that  we  have 
begun  to  develop  physiological  tests  derived  from  them.  These  involve  the  development  of  novel 
stimuli  in  which  short  segments  of  opposite  contrast  bars  are  superimposed  onto  bnger  bars  at 
preferred  orientations,  and  then  extended  orthogonally.  The  end  result  is  a  cross  composed  of 
(say)  a  dark  horizontal  bar  and  a  white  vertical  bar  on  a  gray  background.  Such  tests  will  permit 
us  to  understand  significantly  more  about  subunit  interactions  in  receptive  fields,  and,  if  our 
predictions  are  substantiated,  will  force  a  rather  different  view  of  receptive  fields  in  early  vision. 

The  system  for  generating  the  relevant  stimuli  was  completed  this  summer  as  a  result  of  a 
student  exchange  between  McGill  University  and  the  University  of  British  Columbia,  and  initial 
testing  has  now  begun  under  Cynader's  direction.  Data  from  the  first  two  experiments  is  included 
in  Figs.  1  and  2,  and  shows  the  response  of  the  cell  to  a  bar  of  preferred  contrast  with  an  opposite 
contrast  blob  at  the  center.  The  horizontal  axis  is  contrast  of  the  blob  (same  to  the  right,  opposite 
to  the  left  of  center)  vs.  response.  It  is  still  too  early  to  draw  any  conclusions  from  these  data, 
but  we  expect  to  have  a  lot  to  report  in  the  6  month  brief  progress  report. 


3.  Modeling  of  Endstopping  and  Curvature 

Our  other  main  area  of  collaboration  has  been  the  modeling  of  endstopped  neurons  in  the 
visual  cortex,  with  an  examination  of  how  well  they  might  respond  to  curved  stimuli.  Recall 
that  our  model  involves  a  rectified  "difference”  between  the  responses  of  two  cells  with  oriented 
receptive  fields,  one  large  and  the  other  small.  Our  model  now  includes  complex  cell  components, 
and  a  very  elaborate  simulation  of  it  has  been  completed.  Two  new  papers,  under  review  for 
Visual  Neuroscience,  are  included  in  Appendices  III  and  IV. 


4.  Motion  and  MT 

We  have  begun  to  extend  our  theoretical  efforts  into  the  motion  domain,  and  to  extend  our 
modeling  efforts  into  area  MT.  After  studying  the  treatment  of  discontinuities  (Zucker.  S.W., 
Iverson,  L.,  and  Hummel,  R.,  Coherent  compound  motion:  Corners  and  non-rigid  configurations, 
Neural  Computation,  1990,  2,  44  -  57),  we  have  begun  to  address  the  question  of  how  the  visual 
system  can  obtain  reliable  estimates  of  the  optical  flow  field  and  how  this  information  can  be 
used  to  obtain  coarse  estimates  of  shape.  Substantial  progress  has  been  made;  see  Appendix  V. 


5.  Toward  a  Computational  Theory  of  Shape 

We  view  the  key  bottleneck  to  high-level  vision  to  be  an  analysis  of  shape,  and  thus  have 
begun  to  address  the  question  of  how  to  decompose  a  contour  into  parts  from  a  theoretical 
perspective.  An  early  overview  of  the  theory  was  included  in  last  year’s  progress  report;  we  now 
include  recent  theoretical  results  in  Appendix  VI. 

In  addition  to  our  theoretical  studies,  we  have  also  begun  to  examine  psychophysical  impli¬ 
cations  of  the  theory.  In  particular,  we  have  discovered  new  classes  of  stimuli  in  which  a  single 
parameter  controls  a  smooth  variation  from  one  shape  into  another;  examples  are  included  in  Fig. 
3  (can  you  locate  the  different  “hourglass”?)  where  neck  width  is  the  parameter,  and  in  Fig.  4, 
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6.  Summary  and  Conclusions 


(can  you  locate  the  "worm"  in  the  field  of  sausages?)  where  phase  angle  is  the  parameter.  Early 
data  are  also  included,  showing  the  time  to  locate  the  singular  shape.  Since  the  theory  predicts 
which  shapes  are  most  similar,  the  psychophical  experiments  are  based  on  the  tenet  that  more 
similar  shapes  should  take  bnger  to  discriminate  between.  Clearly,  although  the  shape  parameters 
vary  smoothly,  the  above  prediction  seems  to  be  verified;  there  is  an  abrupt  parameter  value  at 
which  time  increases  substantially.  Again,  more  psychophysics  needs  to  be  done,  and  results  need 
to  be  quantified.  This  will  be  reported  on  at  6  months. 

6.  Summary  and  Conclusions 

It  should  be  clear  from  the  above  overview  that  our  laboratories  have  been  extremely  busy, 
with  projects  in  receptive  field  modeling,  endstopping,  motion,  and  shape  all  underway.  There 
are  in  addition  a  number  of  other,  more  theoretical  projects  as  well,  such  as  the  use  of  potential 
functions  to  fill  the  gulf  between  local  and  global  visual  representations  (included  as  Appendix  VII 
for  completeness).  While  these  projects  may  seem  rather  diverse  when  presented  in  this  fashion, 
we  invite  you  to  visit  our  laboratories  this  year,  to  sample  the  activity  more  personally.  This  would 
give  us  the  opportunity  to  discuss  the  larger  view  of  vision  that  we  have  been  evolving,  and  which 
inspires  each  of  these  more  technical  activities. 
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Appendix  I 


Logical/Linear  Operators  for  Measuring  Orientation  and  Curvature,  Technical  Report  TR-CIM- 
90-06,  McGill  Research  Center  for  Intelligent  Machines,  Montreal. 
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Abstract 

We  propose  a  language  which  unifies  aspects  of  linear  operator  theory  and  boolean  logic  to 
define  logical/linear  (L/L)  operators.  A  family  of  these  operators  appropriate  for  measuring  the 
low-order  differential  structure  of  image  curves  is  developed.  These  L/L  operators  are  derived  by 
decomposing  a  linear  model  into  logical  components  to  ensure  that  certain  logical  preconditions 
for  the  existence  of  an  image  curve  are  upheld.  The  resulting  operators  allow  for  coarse 
measurement  of  curvilinear  differential  structure  (orientation  and  curvature)  while  successfully 
segregating  edge-  and  line-like  features.  By  thus  reducing  the  incidence  of  false-positive  responses, 
these  operators  are  a  substantial  improvement  over  (thresholded)  linear  operators  which  attempt 
to  resolve  the  same  class  of  features.  Furthermore,  these  logical  decomposition  techniques 
naturally  overcome  the  need  for  the  ad  hoc  post-processing  linear  methods  normally  rely  on. 
Finally,  evidence  is  advanced  which  would  indicate  that  these  operators  are  appropriate  models 
for  end-stopped  and  non-end-stopped  simple  cells  in  primary  visual  cortex. 

Index  items:  Edge  detection,  feature  extraction,  image  processing,  computer  vision,  non-linear 
operators. 
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1.  Introduction 


There  is  no  shortage  of  so-called  "edge-detectors"  and  "line-detectors"  in  computer  vision.  These 
operators  are  intended  to  respond  to  lines  and  edges  in  intensity  images.  Many  different  designs 
have  been  proposed,  based  on  a  range  of  optimality  criteria  (e.g  [Heuckel,  1971;  Canny,  1986]) 
and,  in  the  end,  many  of  these  designs  exhibit  properties  in  common  with  biological  vision 
systems  [Jones  and  Palmer,  1987].  While  this  agreement  between  mathematics  and  physiology  is 
encouraging,  there  is  still  dissatisfaction  with  these  operators — despite  their  'optimal'  design,  they 
do  not  work  sufficiently  well  to  support  subsequent  analysis.  Part  of  the  problem  is  undoubtedly 
the  myopic  perspective  to  which  such  operators  are  restricted,  suggesting  the  need  for  more 
global  interactions  [Zucker  et  al,  1988].  But  we  believe  that  more  can  be  done  locally,  and  that 
another  significant  part  of  the  problem  stems  from  the  types  of  models  on  which  the  operators 
are  based  and  the  related  mathematical  tools  that  have  been  invoked  to  derive  them.  In  this 
paper  we  introduce  an  approach  to  operator  design  that  differs  significantly  from  the  standard 
practice,  and  illustrate  how  it  can  be  used  to  design  non-linear  operators  for  locating  lines  and 
edges.  Finally,  relating  these  non-linearities  back  to  physiology  suggests  explanations  for  some 
of  the  characteristics  of  simple  cells  left  unexplained  by  linear  models. 

A  standard  model  used  in  the  design  of  edge  operators  involves  two  components:  an  ideal 
step  edge  plus  additive  gaussian  noise.  This  model  was  proposed  in  one  of  the  first  edge  detector 
designs  [Herskovitz  and  Binford,  1970],  and  has  continued  through  the  most  recent  [Canny, 
1986;  Deriche,  1987].  Thus  it  is  no  surprise  that  the  solution  resembles  the  product  of  two 
(operator)  terms,  one  to  smooth  the  noise  (e.g.  a  gaussian)  and  the  other  to  locate  the  edge  (e.g.  a 
derivative  or  a  step  function). 

While  some  of  the  limitations  of  the  ideal  step  edge  model  have  been  addressed  elsewhere 
(e.g.  [Horn,  1977;  Leclerc  and  Zucker,  1984]),  a  perhaps  more  important  limitation  of  the  operator 
design  has  not  been  considered.  It  is  assumed  that  in  viewing  a  small  local  region  of  the  image, 
only  a  single  section  of  one  edge  is  being  examined.  This  may  be  a  valid  simplification  in  some 
continuous  limit,  but  it  is  definitely  not  valid  in  digital  images.  Many  of  the  systematic  problems 
with  edge  and  line  detectors  occur  when  structure  changes  within  the  local  support  of  the  operator 
(e.g.  several  edges  or  lines  coincide).  Since  these  singularities  are  not  dealt  with  by  the  noise 
component  of  the  model  either,  the  linear  operator  behaves  poorly  in  their  vicinity. 

In  particular,  curve-detecting  operators  are  usually  designed  to  respond  if  a  certain  intensity 
configuration  exists  locally  (see  Figure  1  (a)).  A  signal  estimation  component  of  the  operator  is  then 
incorporated  in  the  design  to  filter  local  noise  (Figure  1  (b)).  However,  significant  contrast  changes 
are  rarely  noise — they  are  more  likely  to  be  the  result  of  a  set  of  distinct  objects  whose  images 
project  to  coincident  image  positions  (Figure  1(c)).  An  operator  which  claims  to  'detect'  or  'select' 
a  certain  class  of  image  features  must  continue  to  do  so  in  the  presence  of  such  confounding 
information. 

We  propose  that  image  operators  should  be  designed  to  respond  positively  to  the  expected 
image  structure,  and  to  not  respond  at  all  when  such  structure  is  not  present.  Simple  linear  operators 
achieve  the  first  of  these  goals,  but  in  order  to  fulfill  the  second  we  must  incorporate  a  more 
direct  verification  of  the  logical  preconditions  for  a  given  feature  into  the  operator  itself.  We 
accomplish  this  by  decomposing  the  linear  operator  into  components  which  correspond  to  the 
logical  assertions  of  the  preconditions.  When  the  expected  image  structure  is  present,  a  boolean 
combination  of  these  responses  produce  a  linear  response,  whereas  if  any  of  the  logical  conditions 


2.  INTRODUCTION 


Figure  1:  A  set  of  potential  image  curve  configurations  which  must  be  considered 
in  the  design  of  operators.  An  ideal  image  (a)  of  a  black  curve  on  a  white 
background;  a  noisy  image  (b)  of  a  lower-contrast  version  of  the  same  curve;  an 
obscured  version  (c)  of  the  ideal  image.  The  oval  in  each  image  represents  the 
spatial  support  of  a  local  operator. 

are  violated  the  reponse  will  be  suppressed  non-linearly.  Because  these  operators  unite  elements 
from  boolean  logic  and  linear  operator  theory,  we  refer  to  them  as  logical  [linear  (L/L)  operators. 

1.1  Image  Curves 

For  consistency  we  shall  adopt  the  following  terminology.  Edges  are  the  curves  which  separate 
lighter  and  darker  areas  of  the  image — the  perceived  discontinuities  in  the  intensity  surface;  Lines 
are  those  curves  which  might  have  been  drawn  by  a  pen  or  pencil  (sometimes  referred  to  as  bars 
in  other  work  [Marr,  1982]).  Image  curves  may  be  either  of  these.  Two  independent  properties 
describe  these  image  curves;  their  structures  along  the  tangential  and  the  normal  directions. 
Tangentially,  lines  and  edges  are  similar;  it  is  the  cross-sectional  structure  that  differentiates 
them. 

Formally,  let  /:  1R2  —  IR  be  an  analytic  intensity  surface  (an  image)  and  a:S  =  (.so,  $i )  —  1R2 
a  smooth  curve  parametrized  by  arc  length  (see  Figure  2).  The  orientation  6(s)  is  the  direction  of 
the  tangent  r(s),  a  unit  vector  in  the  direction  of  a'(s),  and  the  normal  vector  n(s)  is  a  unit  vector 
in  the  direction  a"(s).  Define  the  cross-section  0,  of  a 

0,(t)  =  l(a(s)  +  tn(s[),  a€S,f€lR.  (1) 

The  three  kinds  of  image  curve  are  then  defined: 

Definition  1:  a  is  a  Positive  Contrast  Line  iff 

3c  >  0:  0,(0)  =  max  0,(6),  s  €  S.  (2a) 

— «5<e 

Definition  2:  a  is  a  Negative  Contrast  Line  iff 

3c  >  0:  /3,(0)  =  min  0,(6),  s  €  S. 


(2b) 
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Figure  2:  An  image  curve  a:S  =  (so,  «i)  — 1 ►  1R2  with  the  tangent  r(s)  and  normal 
n{s). 


Definition  3:  a  is  an  Edge*  iff 


ft  >  0  and  3c  >  0:  |# (0)|  =  max  \/3's(6)\,  s  G  S. 

— t<fi<e 


(2c) 


These  are  the  structural  preconditions  that  define  image  curves:  they  must  hold  over  their 
length  (S)  and  width  (e).  Linear  operators  do  respond  when  these  conditions  are  met.  However, 
they  also  respond  in  characteristic  situations  in  which  the  conditions  are  clearly  not  met.  These 
responses  are  referred  to  as  false-positives.  The  current  analysis  will  focus  on  a  mechanism  for 
avoiding  three  kinds  of  false-positives  typical  of  linear  operators: 

1.  Confusion  between  lines  and  edges:  Lines  and  edges  are  differentiated  in  terms  of  their  cross- 
sections.  For  accurate  identification  the  logical  conditions  on  the  cross-section  must  be 
satisfied,  and  in  each  case  we  will  show  that  a  linear  operator  tests  them  incompletely. 

2.  Merging  or  interference  between  nearby  curves:  The  local  continuity  of  image  curves  is 
important  for  resolving  and  separating  nearby  features.  Linear  operators  interfere  with 
testing  continuity  by  filling  in  gaps  between  nearby  features. 

3.  Smoothing  out  discontinuities  or  failing  to  localize  line-endings:  The  locations  of  the  discontinu¬ 
ities  and  end-points  of  a  curve  are  fundamental  to  higher-level  descriptions  [Koenderink  and 
van  Doom,  1976;  1982]  and  [Koenderink,  1984].  Linear  operators  systematically  interfere 
with  their  localization  by  arbitrary  smoothing. 


‘Note  that  the  definition  of  a  line  is  sign  specific,  whereas  an  edge  is  defined  as  a  locus  of  local  maxima  or  minima. 
There  is  no  conceptual  difference  between  edges  that  go  hum  light  to  dark  moving  towards  their  centre  of  curvature 
and  those  of  the  opposite  contrast;  they  are  both  edges. 
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2.  A  Logical/Linear  Framework  for  Line  Operators 


The  three  qualitatively  different  kinds  of  image  curves  defined  in  §1.1  imply  three  distinct  sets 
of  preconditions  for  the  existence  of  an  image  curve.  As  noted  previously,  the  curve  description 
process  must  respect  these  distinctions.  Focusing  for  the  moment  on  the  line  condition  of 
Definition  1*,  we  begin  by  adopting  an  oriented,  linear  line  operator  similar  to  the  one  described 
in  [Canny,  1986]. 

Canny  adopted  the  assumption  of  linearity  to  facilitate  noise  sensitivity  analysis,  and  relied 
on  post-processing  to  guarantee  locality  and  selectivity  of  response.  He  arrived  at  a  line  operator 
whose  cross-section  is  similar  to  a  gaussian  second-derivative,  and  an  edge  operator  similar  to 
a  gaussian  first  derivative.  Neurophysiologists  [Movshon  et  ah,  1978;  Schumer  and  Movshon, 
1984;  Jones  and  Palmer,  1987]  and  psychophysicists  [Shapley  and  Lennie,  1985]  have  adopted 
such  linear  models  to  capture  many  of  the  functional  properties  of  the  early  visual  system,  and 
they  are  relatively  easy  to  analyse  mathematically  (e.g.  using  Fourier  analysis).  These  models 
are  also  attractive  from  a  strictly  computational  point  of  view  because  they  exhibit  most  of  the 
properties  required  of  a  measurement  operator  for  image  curves.  However,  they  also  exhibit  the 
false-positive  responses  described  above  (partially  shown  in  Figures  5  and  6). 

To  limit  these  false-positives,  we  relax  the  assumption  of  linearity  and  test  the  necessary  local 
conditions  explicitly.  This  is  accomplished  by  decomposing  the  operator  into  components  which 
measure  the  conditions  to  be  tested.  The  resulting  operator  will  appear  to  be  linear  as  long  as  all 
of  these  conditions  are  fulfilled,  and  thus  for  many  of  the  standard  'probe  stimuli'  used  by  visual 
neurophysiologists.  We  will  refer  to  these  operators  as  logical/linear  (L/L)  operators. 


2.1  Logical/Linear  Decomposition 

Consider  the  necessary  conditions  for  a  decomposition  which  will  be  specific  to  the  design 
preconditions  but  still  retain  some  of  the  desirable  properties  of  a  linear  operator. 


Definition  4:  A  non-linear  operator  is  said  to  be  a  Logical/Linear  operator  for  a  given  feature  iff 

1.  It  consists  of  a  linear  operator  which  is  decomposed  into  a  set  of  linear  component  operators  whose 
sum  is  identical  to  the  initial  operator.  These  components  correspond  to  measurements  of  the  logical 
conditions  for  the  feature's  existence. 

2.  The  overall  operator  response  is  positive  only  if  the  logical  conditions  for  the  desired  feature  are 
fulfilled.  When  this  is  so,  the  operator  should  act  identically  to  the  original  linear  operator. 


Thus,  given  a  linear  operator  which  is  conditionally  optimal  (e.g.  in  terms  of  noise  immunity 
and  localization)  a  linear/ logical  decomposition  of  this  should  retain  such  optimality  as  long  as 
the  conditions  on  the  component  responses  hold. 

The  first  condition  above  suggests  that  the  required  decomposition  is  a  partition: 


’That  this  analysis  is  applicable  to  the  other  cases  will  be  shown  in  $3.1.1 
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Definition  5:  For  a  function  f:X->TR.,aset  of  functions  II  =  { ,  ni, . . . }  where  7rt:  X  -*  IR  is  a 
partition  of  /  iff  for  all  x  e  X 

£  7Tj(x)  =  f{x).  (3) 

irj€n 


Many  such  partitions  are  clearly  possible;  we  must  choose  the  components  II  such  that  they 
embody  the  required  logical  features. 

The  combinators  to  fulfill  the  second  condition  of  Definition  4  can  be  derived  from  a  mapping 
of  the  real  line  to  logical  values.  Assume  that  positive  operator  responses  represent  confirmation 
of  a  logical  condition  (logical  true )  and  that  negative  responses  represent  rejection  (logical  false). 
To  derive  the  numeric=>logical  mapping,  we  adopt  the  principle  that  all  confirming  evidence 
should  be  combined  if  the  logical  condition  holds,  and  contradictory  evidence  combined  if  the 
condition  fails.  This  leads  to  the  following  set  of  logical/linear  combinators: 


Definition  6:  The  basic  set  of  Logical/Linear  Combinators  b,  absolute  value,  and  4>  (rectification) 
are  given  by 


x  by 


A 


x  +  y, 
y . 
x , 

x  +  y, 


if  x  >  0  A  y  >  0; 
if  x  >  0  A  y  <  0; 
if  x  <  0  A  y  >  0; 
if  x  <  0  A  y  <  0. 


x  y  =  { 


x  +  y, 

x, 

V , 

l  x  +  y, 


if  x  >  0  A  y  >  0; 
if  x  >  0  A  y  <  0; 
if  x  <  0  A  y  >  0; 
if  x  <  0  A  y  <  0. 


1*1 


a  f  x,  ifx  >  0; 
\  -x,  ifx<0. 


if  x  >  0; 
if  x  <  0. 


(4a) 


(4b) 


Observe  that  |x|  =  x  b  -x  and  4>{x)  =  -(0  b  ab¬ 
using  these  combinators,  define  a  generative  syntax  for  L/L  expressions. 


Definition  7:  A  Logical/Linear  Expression  of  the  real  variables  v  =  {  vi, . . . ,  vn }  is  any  expression 
e  in  the  language  E  defined  by  the  following  grammar  (for  the  as  being  real  constants): 


E  -*  Vi  E 

E  — ♦  E  b  E  E 

E  -*  \E\  E 


aE 
E$  E 
<KE) 


Consider  the  universe  of  vectors  U  in  IR"  excluding  the  axes* 

U  =  {t>€lRnIt>i^0,l<  »<n>  (5) 

and  the  subspaces  {e}+  =  {t>etf|e>0}. 

‘For  real-valued  variables,  the  exclusion  of  the  axes  needed  to  demonstrate  logical  equivalence  is  not  problematic 
because  of  the  singularity  of  this  restriction. 
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2.  A  LOGICAL/UNEAR  FRAMEWORK  FOR  LINE  OPERATORS 


Theorem  1:  (Logical)  For  the  language  of  L/L  expressions  E,  the  set  of  all  sets  { e }+  and  their 
complements  {e}+  =  U  -  {e}+  (for  e  £  E)form  a  Boolean  algebra  with  meet  k,join  $  and  complement 

The  following  equivalences  can  be  derived  directly  from  Definition  6,  for  all  e\ ,  e2  e  E: 


{  ~el  }+  =  {el}+» 

(6a) 

{  A  ^2  }+  =  {ei  }+  n{e2  }+, 

(6b) 

{ei  ^  e2  }+  =  {  ei  }+  U  {  e2}+. 

(6c) 

It  is  easy  to  verify  that  these  sets  form  a  field  with  the  help  of  the  equivalences  above  (e.g.  the 
equivalence  of  A  and  n  ensures  that  if  X  and  Y  are  members  then  X  A  Y  is  also).  Furthermore, 
these  meet,  join  and  complement  operators  are  clearly  isomorphic  with  the  standard  set-theoretic 
n,  u  and  complement.  The  further  observation  that  0  and  U  are  the  bounds  of  this  field  ensure 
that  this  system  is  a  Boolean  algebra.  ([Sikorski,  I960],  p.  3)  | 


The  following  equivalences  can  also  be  derived  directly  and  are  useful. 

(“«}+  =  {e}+ifa>0  {ae}+  ={7}+ifa<0 

{  lel  }+  =  U  {*(*)}+  =  {e}+- 

To  expose  the  linearity  inherent  in  these  operators,  it  is  necessary  to  consider  the  2n  minimal 
polynomials 

Pj  -  91  A  92  A  •  •  •  A  Qn  (7) 

where  <7,  =  v,  if  bit  i  in  the  binary  representation  of  j  is  zero,  <?,  =  -v,-  if  bit  i  is  one. 


Theorem  2:  (Linear)  Any  L/L  expression  e  is  linear  within  the  subspace  {  Pj }+  of  any  minimal 
polynomial  Pj. 

Any  Boolean  polynomial  can  be  equivalently  expressed  as  the  join  of  minimal  polynomials  or 
the  lower  bound  0  (iBirkhoff  and  MacLane,  1977],  p.  370).  Thus  {  e }+  can  be  expressed  as  the  V 
of  a  group  of  such  minimal  polynomials  (the  disjunctive  canonical  form  (DCF)  of  e).  Without  loss 
of  generality,  consider  a  particular  such  polynomial  Pj.  Noting  that  every  element  v,  in  a  vector 
v  6  {Pj }+  has  a  fixed  sign.  Definition  6  guarantees  that  A  is  linear  on  the  subspace  defined 
by  {  Pj }+  (for  fixed  sign  arguments,  the  branch  chosen  in  the  A  is  fixed).  Thus,  any  minimal 
polynomial  Pj  is  linear  on  { P}  }+. 

Consider  now  the  DCF  of  e.  We  know  that  each  Pj  in  this  DCF  is  both  linear  and  of  constant 
sign  on  {  Pj }+.  By  the  same  reasoning  as  for  A  above,  we  can  state  that  is  linear  if  its  arguments 
have  constant  sign,  and  thus  the  DCF  of  e  is  a  linear  combination  of  expressions  which  are 
guaranteed  linear  on  {  P} }+.  Therefore,  e  is  also  linear  on  every  {  Pj }+.  | 


Thus,  the  proposed  combinators  satisfy  the  goals  of  Definition  4,  and  have  both  logical  and 
linear  properties.  We  will  rely  on  both  of  these  properties  in  constructing  the  L/L  image  operators. 
Refer  to  Appendix  A  for  a  more  detailed  analysis  of  the  implications  of  these  L/L  constructs. 
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3.  Designing  L/L  Operators  for  Image  Curves 
3.1  Testing  Lateral  Extrema 

We  begin  by  localizing  the  cross-section  of  a  positive  contrast  line  (Definition  1).  A  necessary 
condition  for  the  existence  of  such  a  line  is  a  local  extremum  in  intensity  (Figure  3  is  a  display 
of  typical  ID  cross-sections  of  these  intensity  phenomena),  we  will  first  consider  the  operator 
structure  perpendicular  to  its  preferred  direction.  This  specializes  to  the  prior  problem  of  locating 
extrema  in  the  cross-section  (3t. 

A  local  extremum  in  a  one-dimensional  signal*  0(x)  exists  only  at  those  points  where 

/3'(x)  =  0  and  /3"(x)  ^  0.  (8) 

Estimating  the  location  of  such  zeroes  in  the  presence  of  noise  is  normally  achieved  by  locating 
zero-crossings,  thus  in  practice  these  conditions  become 

/3'(x  -  e)  >  0  and  / 3'(x  +  e)  <  0  and  /3"(x)  <  0  (9) 

An  operator  which  can  reliably  restrict  its  responses  to  only  those  points  where  these  conditions 
hold  will  only  respond  to  local  maxima  in  a  one-dimensional  signal. 

A  set  of  noise-insensitive  linear  derivative  operators  (or  'fuzzy  derivatives'  [Koenderink  and 
van  Doom,  1987])  are  the  various  derivatives  of  a  Gaussian  envelope. 


G.(X)  =  -L- 

V2na 

(10a) 

g;(x)  =  -£g,(x), 

(10b) 

C"M  =  G.(x). 

(10c) 

When  convolved  over  a  one-dimensional  signal  these  give  noise-insensitive  estimates  of  the 
derivatives  of  the  signal.  These  smooth  derivatives  of  the  signal  are 

&(*)  =  /3'(x)  *  Gff(x)  =  (3{x)*GUx),  Ola) 

#,'(*)  =  /3"(*)*G„(x)  =  0(*)*G"(x).  (lib) 


Theorem  3:  (3g(x  -  f)  >  0  and  (3'g{x  +  e)  <  0  and  /3"(x)  <  0  are  sufficient  conditions  for  a  local 
maximum  in  the  signal  /3(x). 

The  identities  of  (11)  show  that  these  conditions  are  necessary  and  sufficient  for  the  existence 
of  a  local  maximum  in  f3a{x)  =  f3(x )  *  G*(x).  The  maximum  principle  for  the  heat  equation 
((Protter  and  Weinberger,  1984],  p.  161)  implies  that  convolution  by  a  gaussian  cannot  introduce 
new  maxima.  Thus  the  above  conditions  imply  the  existence  of  a  maximum  in  /3(x).  | 


‘Assume  f)(z)  is  sufficiently  differentiable. 
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(a)  (b) 


Figure  3:  Cross  sections  of  image  lines  and  edges.  A  line  in  an  intensity  image  (a) 
is  located  at  the  peak  of  its  cross-section.  Note  that  this  coincides  with  a  zero  in  the 
derivative  /3'  and  a  negative  second  derivative  /3".  An  intensity  edge  (b)  occurs  at 
peaks  in  the  derivative  &'  of  the  cross-section.  The  derivatives  shown  are  derived 
from  convolution  by  G'  and  G"  operators  with  a  =  3. 


Figure  4:  Approximation  of  (a)  G'  by  the  difference  of  two  G  operators,  and  (b)  G" 
by  the  difference  of  two  G'  operators. 


3.1.  TESTING  LATERAL  EXTREMA 
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This  suggests  a  practical  method  for  locating  maxima  in  a  noisy  one-dimensional  signal. 
Comparing  the  results  of  convolutions  by  derivatives  of  gaussians  will  allow  us  to  determine  the 
points  where  Theorem  3  holds.  The  loci  of  such  points  will  form  distinct  intervals  with  widths 
<  2e.  The  parameter  a  determines  the  amount  of  smoothing  used  to  reduce  noise-sensitivity. 
Using  central  limits,  observe  that: 

/'(*)  =  Jm  !(/(!  +  €)- /(!-,)).  (12) 

Thus  for  the  fuzzy  derivatives  of  /3,  one  would  expect  to  find  that 

+<)-&(*-<))  (13) 

with  the  quality  of  the  approximation  a  function  of  t.  Thus  the  conditions  in  Theorem  3  can  be 
derived  from  the  derivative  /%(x)— a  linear  combination  of  two  points  will  give  /3"(x).  More 
specifically,  we  adopt  the  approximation  G"(x )  as  (g^(x  +  e)  -  G'a{x  -  e))/2e,  where  e/a  <  1. 
Figure  4(b)  shows  that  for  e  =  a/2  this  is  an  excellent  approximation. 

Thus,  convolution  by  G'  allows  testing  of  all  three  conditions  in  Theorem  3  simultaneously. 
Using  the  L/L  combinators  of  §2.1  we  are  now  able  to  define  a  one-dimensional  operator  which 
has  a  positive  response  only  within  a  small  range  of  local  maxima. 


Definition  8:  The  One-Dimensional  Maxima  Operator  is  given  by 

*»(/3)l,  =  (»+(*)*/3(x))a(«7(*)*0(*))  (14a) 


where 


K  =  -GUx  +  t)/2c,  (14b) 

=  G^(x  -  e)/2e.  (14c) 


We  rely  on  a  shorthand  notation  when  describing  operators  as  combinations  of  linear  or 
logical/linear  components  which  are  evaluated  at  all  x;  (14a)  can  be  equivalently  expressed  as 

=  S'-AS'+.  (15) 

Clearly  then  the  key  advantage  of  this  S'*  operator  is  that: 


Lemma  4:  The  response  9C(/3)\X  zvill  be  positive  only  if  there  is  local  maximum  in  (3a  within  the  region 
[x  -  6a,  x  +  6a]. 

By  Theorem  1  we  know  that  >  0  implies  that  both  (16b)  and  (16a)  must  also  be 

positive.  Equations  11  then  imply  that 


*a(x)*P(x)  =  -#,(*  +  0/2f 
¥+(x)*/3(x)  =  0'„(x-e)/2e 


(16a) 

(16b) 
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(a) 


(b) 


(c) 


Figure  5:  Responses  of  L/L  and  linear  maxima  operators  near  step.  Rectified 
responses  of  the  G"  operator  and  divided  G'  operators  are  shown  for  three  step 
edges  with  varying  slopes  of  the  upper  region.  It  can  be  seen  that  the  non-linear 
operator  blocks  the  unwanted  response  near  a  step  which  is  not  also  a  local 
maximum  (a  &  b),  but  that  when  it  is  a  local  maximum  (c)  it  does  respond.  The 
linear  G"  operator,  however,  responds  in  each  of  these  cases,  exhibiting  consistent 
(and  erroneous)  displacement  of  the  peak  response. 


Thus  a  positive  response  requires  that  0'c(x  -  e)  >  0  and  0'o{x  +  c)  <  0,  which  in  turn  imply  the 
presence  of  a  local  maximum  on  (3a  between  x  -  e  and  x  +  e."  ■ 


The  performance  improvement  from  introducing  this  non-linearity  is  considerable.  The 
linear  operator  exhibits  consistent  patterns  of  false  positive  responses.  The  simplest  example  is 
the  response  near  a  step  (see  Figure  5).  The  linear  operator  displays  a  characteristic  (false)  peak 
response  when  the  step  is  centered  over  one  of  the  zeroes  in  the  operator  profile.  The  logical/linear 
A  operation  prevents  this  error  since  both  G'  halves  of  the  operator  register  derivatives  in  the 
same  direction  and  so  do  not  fulfill  the  conditions  of  (9).  The  L/L  operator  will  respond  positively 
only  in  the  case  that  the  slope  above  the  step  is  negative  (i.e.  only  when  the  transition  point  is  also 
a  local  maximum). 

3.1.1  Other  Curve  Types 

The  extension  of  this  analysis  to  the  other  curve  types  in  §1.1  is  straightforward.  The  most  obvious 
observation  is  that  the  above  analysis  can  be  repeated  entirely  with  a  simple  change  of  sign  so  as 
to  be  specific  for  one-dimensional  minima,  which  correspond  to  negative  contrast  lines. 

Slightly  more  complicated  is  the  case  for  edges,  where  two  related  conditions  must  be 
combined  (Definition  3).  For  this  case,  the  following  conditions  (analogous  to  equation  (8))  must 
be  verified 

0\x)  >  0  and  (3"{x)  =  0  and  /3"'(x)  <  0  (17a) 

or 

0\x)  >  0  and  0"(x  -  c)  >  0  and  /3"(x  +  e)  <  0  and  0"\x)  <  0  (17b) 


‘Observe  that  although  the  local  maximum  In  f}0  is  guaranteed  to  fall  within  this  region,  the  corresponding 
maximum  in  f)  is  not  necessarily  so  restricted.  Qualitatively  however,  we  can  rely  on  the  observation  that  the  maxima 
for  a  signal  will  converge  on  the  centroid  of  that  signal  under  heat  propagation  (or  as  we  convolve  with  larger  and 
larger  gaussians).  Considering  the  features  of  /)  in  isolation  then,  we  can  state  that  the  smoothing  will  cause  the  location 
of  the  local  maximum  in  to  shift  towards  the  centroid  of  the  local  intensity  distribution. 
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Mirroring  the  analysis  above,  the  verification  of  these  conditions  can  be  realized  in  an  L/L 
operator  by  defining  the  components 


V  =  GUs) 

(18a) 

*2~  =  G"(x  -  e) 

(18b) 

n  =  -  C"(x  +  e) 

(18c) 

and  the  combination 

*  =  A  ^2  4  ®2 

(18d) 

or,  equivalently. 

$  =  $•  A(«*  -  ®~)  *(¥•  -  $+) 

(19) 

where  3'-  and  '3'+  are  as  defined  in  (14). 

Supporting  this  use  of  the  third  derivative  as  a  measure  of  edge  confidence  is  the  observation 
that  a  blurred  step  edge  has  vanishing  even  derivatives  and  positive  odd  derivatives.  The 
description  of  an  edge  adopted  in  (17)  is  clearly  consistent  with  this  observation.  It  is  also  more 
selective  than  the  'zero-crossing  of  a  second-derivative/  [Marr  and  Hildreth,  1980]  which  is  only 
one  of  the  logical  preconditions  which  this  operator  depends  on  for  a  response.  One  can  therefore 
expect  less  of  a  problem  of  non-edge  signals  generating  edge-like  responses  with  this  operator 
than  with  those  less  specific. 

3.2  Testing  Tangential  Continuity 

So  far  only  the  cross-sectional  (normal)  image  structure  (J3S)  has  been  discussed.  In  order  to 
extend  this  result  to  two-dimensions,  we  must  examine  the  tangential  (curvilinear)  structure  of 
the  curves  (a).  By  (2)  we  must  verify  the  local  connectedness  of  candidate  curves.  In  addition, 
the  extraction  of  orientation-specific  measures  was  deemed  essential  for  further  processing.  In 
this  section,  these  problems  will  be  addressed  by  imposing  a  further  tangential  structure  on  the 
operator. 

We  will  follow  the  same  course  as  for  the  cross-section — first  a  linear  structure  is  proposed 
which  will  be  decomposed  to  reveal  linear  measurement  operators  for  the  components  of  the 
logical  preconditions.  Assume  initially  that  the  intensity  variation  along  a  curve  is  everywhere 
smooth  and  corrupted  only  by  additive  gaussian  noise.  This  suggests  that  the  response  should 
be  noise-filtered  with  a  linear  gaussian  operator.  Incorporating  the  prior  conditions  of  §3.1,  we 
arrive  at  the  first  two-dimensional  candidate  operator: 


Definition  9:  A  preliminary  Two-Dimensional  Positive  Contrast  Line  Operator  is  given  by 
9  =  bV+  where  the  lateral  components  "  and  ¥+  are  given  by 

*■(*.»)  =  -  + €) 

*+(*,V)  =  c;,(»  -c)G,,(*). 


This  operator  is  specific  to  positive  contrast  lines  with  local  orientation  0  =  0. 


(20a) 

(20b) 
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Figure  6:  The  signal  (top)  is  the  parallel  cross-section  of  an  image  line  near  the 
discontinuous  termination  of  the  line  (the  endline).  Note  that  the  linear  operator 
(middle)  exhibits  a  smooth  attenuation  of  response  around  the  line  ending.  The 
goal  (bottom)  is  an  operator  whose  response  attenuates  abruptly  at  or  near  the 
endline  discontinuity. 


Figure  7:  Schematic  of  the  half-field  decomposition  and  line  endings.  The  elliptic 
regions  in  each  figure  represent  the  operator  positions  as  the  operator  moves 
beyond  the  end  of  an  image  line.  In  (a)  the  operator  is  centered  on  the  image 
line  and  the  line  exists  in  both  half-fields.  In  (b)  the  operator  is  centered  on  the 
end-point  and  whereas  the  line  only  exists  in  one  half-field,  the  other  half-field 
contains  the  end-point.  In  (c)  the  operator  is  centered  off  the  line  and  the  line  only 
exists  in  one  half-field. 
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Note  that  this  operator  assumes  both  local  continuity  ol  a,  and  slow  variation  in  contrast 
along  the  tangential  direction.  The  indiscriminate  smoothing  of  the  gaussian  G0x  provides  a  post 
hoc  guarantee  of  these  assumptions  by  smoothing  out  variation  in  the  image.  It  will  remove  both 
orientation  and  contrast  discontinuities  (see  Figure  6)  in  the  image  curve,  thus  violating  the  third 
criterion  of  §1.1.  The  local  continuity  of  the  curve  must  be  verified  prior  to  smoothing. 

To  resolve  this,  consider  the  continuity  of  the  curve  a(s). 


Definition  10:  The  curve  a  is  C1  Continuous  at  so  iff 

lim  a(s)  =  lim  a(s)  =  a(«o) 
lim  9(s)  =  lim  0(s)  =  0(so) 

s— »*0—  »— »so+ 


(21a) 

(21b) 


Thus,  if  the  extensions  of  a  curve  on  either  side  of  a  point  oc(sq)  both  exist  and  are  similarly 
oriented,  then  the  hypothesis  of  local  C\  continuity  at  so  is  confirmed. 


Definition  11:  An  operator  is  End-Line  Stable  if  it  responds  positively  only  when  centered  on  a 
continuous,  connected  region  of  an  image  curve. 


Representing  the  intensity  variation  along  the  curve  q  as  a  function  of  the  arc-length  I0(s), 
an  ideal  line-ending  (or  beginning)  is  a  step  in  intensity  at  s  =  0.  End-line  stability  requires  that 
the  operator's  response  ^(/Q)|,  is  non-positive  for  all  s  <  e,  and  positive  for  s  >  e*  Figure  7 
demonstrates  that  this  can  be  achieved  by  separately  considering  the  behaviour  of  the  curve  in 
both  directions  around  the  operator  centre. 

We  therefore  adopt  a  partition  which  divides  the  operator  kernel  into  regions  along  its  length. 
Using  the  step  function 


1 1 ,  if  x  >  0; 

1 0,  otherwise. 


(22) 


a  partition  of  F{x)  around  0  is  given  by 


F~(x)  =  F{  i)E(-x)  F+(  x)  =  F(  x)E(x) 


(23) 


Note  that  F~(x)  +  F+(x)  =  F(x)  for  all  x,  as  required.  A  smooth  (and  more  stable)  partitioning 
operator  is  described  in  Appendix  A. 

Applying  this  partition  to  the  gaussian  smoothing  kernel,  we  obtain  the  two  half-field 
operators  (see  Figure  8(a)) 

G;(x)  =  Go(x)  E(-x)  Ct(x)  =  Ge(x)  S(x)  (24) 

Which  are  combined  to  produce  the  end-line  stable  G  operator. 


Th  eorem  5:  The  operator  =  G"  A  G£  is  end-line  stable. 


’This  property  must  also  operate  symmetrically  at  the  other  end  of  the  curve. 
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(a) 


(b) 


Figure  8:  Splitting  of  ID  gaussian  operator  into  components.  Figure  (a)  shows  the 
division  about  the  center  described  by  (24)  and  the  subsequent  analysis.  Figure  (b) 
is  an  example  of  the  extension  of  this  to  four  regions  and  the  addition  of  response 
stabilizers  as  described  in  (27).  The  centers  of  division  (cj)  partition  the  underlying 
area  equally.  The  stabilizers  have  tr'  =  1 .2 and  w  =  i/2. 


Consider  the  component  responses  in  the  neighbourhood  of  the  step  edge  IQ(s)  =  E(s).  The 
response  of  G+  to  this  step  is  given  by 


(C-  »  £)|. 

r  G*(s-T)Z(T)dT 
J—oo 

f  Z(s  -  r)  Go{s  -  t)  dr 
JO 

I'  Z(T)Ga(T)dT 
J— oo 


G(r(r)  dr, 


if  s  >  0. 
if  s  <  0; 


(25) 


The  L/L  AND  of  G+  and  Gj  to  produce  Ga  requires  that  both  component  responses  be  strictly 
positive  for  a  positive  response,  thus  whenever  s  <  0  around  the  step  described  above,  the  G 
response  is  also  zero.  It  is  obvious  that  the  same  analysis  applies  to  the  symmetric  Gj  component 
and  the  1  -  E(s)  step  edge,  which  describes  behaviour  around  the  other  end  of  the  line.  Thus  the 
G  operator  is  end-line  stable  symmetrically  around  a  step  edge.  ■ 


Observe  however  that  the  component  responses  of  the  pure  partition  described  above  are 
unstable  with  respect  to  the  4  combinator.  The  a  <  0  case  in  equation  (25)  produces  a  zero 
response,  which  forces  the  aggregate  response  of  the  A  to  be  zero.  The  proximity  of  this  value 
to  the  singular  point  of  the  4  combinator  ensures  that  the  overall  response  is  very  sensitive  to 
perturbations  in  the  component  responses.  This  can  be  ameliorated  by  adding  a  stabilizer  to  the 
partition,  which  will  force  the  component  responses  negative  where  they  would  otherwise  be 
simply  zero.  While  many  stabilizers  will  do  we  adopt  the  first  derivative  of  a  gaussian  as  perhaps 
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the  "simplest'  smooth  operator  whose  response  is  entirely  negative  on  one  side  of  a  step  edge 
and  entirely  positive  on  the  other. 

This  result  can  be  combined  with  the  approach  of  [Davis  et  al.,  19761,  in  which  a  linear 
operator  is  made  noise-insensitive  by  dividing  it  into  a  number  of  sections  along  its  length  and 
'detecting'  a  line  only  when  more  than  half  of  the  regions  give  a  positive  response  (this  is  also 
similar  to  the  ANDing  of  LGN  afferents  proposed  in  [Marr  and  Hildreth,  19801).  Combining 
these  two  principles  requires  that  the  operator  be  smoothly  divided  into  some  even  number  of 
regions  around  its  centre.  The  partition  of  (23)  can  be  generalized  to  n  >  2  regions  by  combining 
partitions  for  a  strictly  increasing  sequence  of  partition  points  {  c,  |  1  <  i  <  n  }  (S, (x)  is  the 
stabilizer  described  above). 

Si(x)  =  w  G'ci(x  -  ci )  (26) 

f.F(*)(l-£(x -<*))-$(*)  if  i  =  l; 

Fi(x)  =  l  F(x)  (E(x  -  Ct_i)  -  £(x  -  c^)  +  S<_i(x)  -  S<(x)  if  1  <  i  <  n;  (27) 

l  F(x)  £(x  -  cn_!)  +  Sn_i(x)  if  i  =  n. 

Algebraic  combination  will  verify  that  these  F,  partition  the  function  F(x).  The  separation  of  a 
gaussian  into  four  equal  area  components  with  G'  stabilizers  (which  we  will  refer  to  as  the  length 
components  of  the  operator)  is  shown  in  Figure  8(b). 

Consider  a  gaussian  operator  thus  partitioned  into  an  even  n  components  GJ,  by  the  introduc¬ 
tion  of  n  - 1  partition  points  c,  .*  Require  then  that  all  responses  in  one  half-field  and  any  response 
in  the  other  half-field  be  positive.*  This  condition  can  be  stated  as 

Definition  12:  (Length  Combination  1)  To  ensure  that  the  operator  G*  will  only  respond 
positively  when  centered  on  a  locally  continuous  image  curve,  combine  the  length  components  of  the 
operator  as  follows 

G;  =  ((GU...AG^)^(G”/2+,^...^G”)) 

$  ((Gi  $  . . .  *  G”/2)  A  (G"^2+1  *  . . .  A  G"))  .  (28) 


Note  that  this  combination  of  the  length  components  is  end-line  stable  (it  is  just  a  generalization 
of  the  2-component  case  of  Theorem  5),  and  that  the  noise  insensitivity  argument  of  [Davis  et  al, 
19761  still  holds,  since  the  conditions  of  (28)  require  that  more  than  half  of  the  Gj,  regions  exhibit  a 
positive  response. 

Although  (28)  is  useful  for  verifying  end-line  stability,  a  logically  more  restrictive  and  simpler 
form  is  used  for  actually  computing  the  operator  responses. 


Definition  13:  (Length  Combination  2) 

G?  =  G^aG^+1  a((GU-”AG;'2-,)^(G^+2A”-4G?))  (29) 


*By  spacing  the  partition  points  such  that  the  area  under  each  of  the  components  G  j  is  equal,  the  contribution  of 
each  component  will  have  the  same  significance  on  the  final  result.  For  a  gaussian  operator  the  partition  points  to 
produce  four  equal  area  components  should  be  { -0.674?, 0,0.674<r }  and  for  six  {  -0.967?,  -0.431?,  0,0.431?, 0.967? }. 

'There  are  a  number  of  choices  of  logical  combination  of  these  responses  which  guarantee  end-line  stability, 
including  the  (Davis  et  al.,  1976]  condition  itself.  This  one  is  perhaps  the  most  structurally  restrictive  while  retaining 
logical  simplicity. 
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This  formulation  simplifies  the  computation  since  each  component  appears  exactly  once.* 

In  order  to  complete  the  analysis,  we  unify  this  tangential  continuity  combination  with  the 
cross-sectional  combination.  Each  of  the  responses  G'a(I)  is  an  L/L  combination  of  the  ith  length 
components  of  (20).  Thus  each  length  component  is  a  test  of  the  lateral  preconditions,  and  the 
tangential  combination  tests  for  continuity  ensuring  end-line  stability. 


Definition  14:  The  Logical/Linear  Operators  $  are  given  by 

®  =  ®n/2  4  ^n/2+I  4  ((®1  4  •  •  -  4  *n/2-l)  ^  (®r/2+2  A  '  • '  4  ®n))  (30) 

where 

a  for  Positive  Contrast  Lines, 

=  -3'“  A  -Vf  for  Negative  Contrast  Lines, 

=  37  A  ($7  -  S'")  4  (**  -  Vf)  for  Edges. 

The  length  components  <!t~,  Vf  are  obtained  by  substituting  (20)  into  (27).  The  3>*  and  3'*  are  obtained 
by  a  similar  extension  of  (18a)  to  two  dimensions  and  length  components. 


3.3  Curvature  and  End-Stopping 

Extending  the  L/L  operators  to  show  curvature  as  well  as  orientation  selectivity  depends  on  the 
work  of  [Dobbins  etuil  1987;  1989],  which  describes  a  computational  model  of  end-stopping  in  the 
primary  visual  cortex.  The  key  finding  is  that  end-stopping  is  related  to  the  measurement  of  local 
curvature,  and  that  models  of  end-stopped  neurons  can  be  derived  by  taking  differences  between 
oriented  linear  operators.  This  model  can  easily  be  expressed  in  the  language  of  L/L  operators, 
and  can  incorporate  the  stable,  L/L  operators  developed  above  as  components. 

Given  two  similarly  oriented  line  operators  S  (short)  and  L  (long)  and  weighting  factors  to 
compensate  for  the  differences  in  their  relative  areas(ios  and  u^),  the  end-stopped  operator  ES  is 
given  by 

ES  =  <*(u*#S)  -  tt^(L))  (31) 

where  4>(x)  is  a  half-wave  rectification.  If  the  excitatory  and  inhibitory  components  are  closely 
matched  in  both  spatial  frequency  bandwidth  (in  the  normal  direction)  and  orientation  bandwidth, 
then  the  ES  operator  will  have  a  characteristic  maximum  response  to  a  curved  line  with  some 
non-zero  curvature*  (see  Figure  9(a)).  Since  we  wish  to  develop  operators  with  signed  curvature 
specificity,  we  will  use  an  L/L  line  operator  as  the  excitatory  component  and  an  L/L  length- 
partitioned  operator  with  a  G'  cross-section  as  the  inhibitory  component.  This  component  is 
laterally  offset  from  center  so  that  its  zero  axis  coincides  with  that  of  one  lateral  component  of 
the  excitatory  operator;  this  places  the  positive  part  of  the  inhibitory  component  directly  under 
the  center  of  the  excitatory  operator,  thus  guaranteeing  maximal  inhibition  for  straight  lines.  The 
odd-symmetry  of  the  inhibition  in  such  an  operator  ensures  that  it  inhibits  response  for  only  one 
sign  of  curvature  (see  Figure  9(b)),  and  thus  results  in  an  ES  line  operator  which  is  tuned  for  both 
magnitude  and  sign  of  curvature. 


*It  should  be  obvious  that  a  condition  for  reconstruction  of  a  partitioned  linear  operator  is  that  each  component  in 
the  decomposition  must  appear  exactly  » times  in  the  reconstruction  expression. 

•in  all  discussion  of  curvature  values  in  this  paper,  curvature  is  expressed  in  units  of  1  /pixels.  A  curvature  of  0.1  is 
therefore  associated  with  a  circle  of  radius  10  pixels. 
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(a)  (b) 


Figure  9:  Curvature  responses  of  operators  resembling  “simple  cells"  of  two 
different  lengths,  and  their  difference,  used  to  construct  an  end-stopped  (ES)  cell: 
(a)  shows  the  responses  for  an  even-symmetric  inhibitory  component,  which  leads 
to  magnitude  selectivity  only;  0?)  shows  that  for  an  odd-symmetric  inhibitory 
component,  both  sign  and  magnitude  of  curvature  are  selected. 


4.  Results 


The  parameters  for  three  base  operators  are  shown  in  Table  1.  The  non-endstopped  operator 
and  the  excitatory  part  of  the  end-stopped  operators  are  laterally  decomposed  with  aa  -  cry  and 
ba  =  e  [see  (20)].  In  addition  the  tangential  decomposition  of  these  parts  into  n  regions  [see  (27)] 
includes  stabilizers  with  a  a'  -  oxj$,  and  w  =  W  [see  (26)].  The  inhibitory  part  of  the  end-stopped 
operators  has  an  odd-symmetric  G'  cross-sec  "ion  laterally  displaced  by  e  and  is  tangentially 
decomposed  in  the  same  way.  The  weights  W  are  the  weights  on  each  part  required  to  normalize 
the  response  for  an  ideal  stimulus  (%100  contrast  line  of  width  2  pixels).  The  positive  curvature 
operators  are  simply  180  degree  rotations  of  the  negative  curvatures. 

Comparing  the  operator  responses  to  the  test  image  in  Figure  11  is  revealing.  All  of  the  L/L 
operators  are  more  specifically  tuned  than  the  linear  (Fig.  12(a)),  and  each  variation  increases 
the  operator  selectivity.  The  cross-sectional  L/L  decomposition  does  indeed  serve  to  localize 
responses  laterally  (Fig.  12(b)),  although  its  strength  in  suppressing  pure  edge  responses  is  not 
apparent  here.  The  tangential  L/L  decomposition  (Fig.  12(c))  does  prevent  the  indiscriminate 
bridging  of  gaps  between  nearby  curves,  and  the  tendency  to  run  off  the  ends  of  curves,  two 
characteristics  of  the  lme  r  operator.  Finally,  the  combination  of  these  two  decompositions 
produces  an  operator  which  behaves  exactly  as  it  should  behave,  with  accurate  localization  in  all 
directions  of  variatic  a  and  stable  behaviour  in  the  vicinity  of  line  endings  and  multiple  lines. 

Note  that  the  only  effect  of  the  L/L  decomposition  is  to  eliminate  certain  of  the  responses 
from  the  linear  operator.  These  examples  show  that  the  L/L  have  been  effective  in  eliminating  all 
but  the  'best'  from  among  the  'potential'  responses.  The  standard  response  to  this  problem  with 
linear  operators  (e.g.  [Canny,  1986])  relies  on  arbitrary  thresholding  and  further  post-processing, 
and  tends  to  leave  the  question  of  how  to  choose  appropriate  thresholds  open  to  the  user. 

Compare  the  L/L  edge  measurements  in  Figure  18(a)  with  Canny's  operator  in  Figure  18(b) 
(both  processes  are  run  at  the  same  scale).  Not  only  is  the  L/L  edge  operator  as  accurate,  it  makes 
explicit  a  great  deal  of  information  which  is  either  unavailable  or  confounding  in  the  Canny 
image.  For  example,  the  cross  in  the  upper  left  is  made  by  two  straight  lines  crossing  each  other. 
This  relationship  is  explicit  in  the  multiple  orientations  exhibited  at  or  near  the  crossing,  whereas 
the  Canny  responses  would  indicate  that  there  is,  in  fact,  a  smooth  transition  from  vertical  to 
horizontal  in  this  neighbourhood.  In  addition,  the  contour  following  of  the  Canny  process  is  not 
only  a  sequential  bottleneck  in  what  should  be  a  purely  parallel  process,  but  also  ensures  that 
the  'noise'  responses  appear  as  well  structured  as  the  'true'  responses,  thus  interfering  with  the 
potential  for  a  later  process  to  effectively  discriminate  between  them.  And  again  we  are  faced 
with  the  same  question  as  with  the  line  example  discussed  above — what  threshold  should  one 
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Table  1:  Parameters  for  a  family  of  L/L  operators 


Figure  10:  Response  profiles  of  a  family  of  curvature-selective  line  operators  timed 
for  curvatures  {  -0.2,  -0.1, 0.0, 0.1 ,0.2}.  These  operators  are  described  in  Table  1. 


Figure  11:  An  artificial  image  used  to  test  the  image  operators.  This  is  an  anti-alia  sed 
grey-scale  image  of  lines  and  curves,  which  represent  the  range  of  orientations  and 
curvatures  represented  by  an  operator  family  and  the  singularities  (line  endings 
and  crossing  lines)  which  generally  cause  problems  for  linear  operators. 
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(d) 


Figure  12:  The  operator  responses  for  the  artificial  test  image.  For  each  test, 
the  responses  shown  are  for  12  orientations  of  a  non-end-stopped  simple  cell, 
(a)  the  positive  operator  responses  for  a  pure  linear  operator,  (b)  the  responses 
for  an  operator  with  only  the  cross-sectional  non-linearity,  (c)  the  responses  for 
an  operator  with  only  the  tangential  non-linearity,  and  (d)  the  responses  for  the 
operator  with  both  non-linearities. 


(0  P  =  8 


(d)*>  =  4 


Figure  13:  The  effect  of  varying  the  L/L  p  on  a  low-contrast  version  of  the  artificial 
test  image,  (a)  Shows  the  low-contrast  image,  and  (b)  the  operator  responses  with 
ideal  L/L  operators  p  =  oo,  (c)  responses  for  p  -  8,  and  (d)  for  p  =  4. 


Figure  15:  L/L  responses  to  a  fingerprint  image. 


Figure  16:  L/L  responses  to  a  remote  sensing  image  of  logging  roads. 


Figure  18:  Comparison  of  edge  responses  for  a  noisy  test  figure  and  (a)  the  L/L 
edge  operators,  (b)  Canny's  operator. 
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use?  Of  course,  the  Canny  responses  could  be  'cleaned-up'  by  making  the  high  threshold  larger, 
but  at  what  price  and  on  what  criteria? 

The  obvious  strengths  of  the  examples  shown  above  are  an  accurate  estimate  of  the  positions, 
orientations  and  curvatures  of  the  image  curves.  Discontinuities  and  bifurcations  in  these  curves 
are  not  smoothed  over,  instead  they  are  represented  as  multiple  tangents  at  the  same  point.  This 
is  clearest  in  the  natural  images.  However,  it  is  clear  that  these  measurements  are  still  somewhat 
sensitive  to  noise  and  local  perturbations,  and  do  not  always  provide  the  best  estimates  of  local 
curvature. 
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5.1  Biological  Implications 

A  number  of  observations  regarding  the  relationships  between  this  model  and  natural  visual 
systems  are  possible.  In  general,  there  is  nothing  in  the  operator  structure  which  is  biologically 
implausible ;  in  fact,  some  attempt  has  been  made  (see  Appendix  A)  to  demonstrate  how  such  a 
model  might  be  instantiated  in  neurophysiological  terms.  The  gross  similarities  between  these 
oriented,  contrast-tuned  operators  and  cortical  simple  cells  have  already  been  noted.  The  model  of 
end-stopping  which  has  been  adopted  to  give  these  operators  curvature-sensitivity  was  motivated 
largely  as  a  biological  model.  However,  a  number  of  more  intriguing  observations  can  be  made. 

Evidence  was  obtained  that  the  tangential  non-linearities  proposed  may  have  a  neurophys¬ 
iological  correlate  by  [Hammond  and  MacKay,  1983].  In  doing  single-cell  recordings  using 
illuminated  bars  as  stimuli,  they  observed  that  a  small  opposite  contrast  region  in  the  midst  of  a 
stimulus  was  able  to  inhibit  the  response  of  a  simple  cell  much  more  effectively  than  would  be 
predicted  by  a  linear  model.  The  opposite  contrast  region  was,  in  effect,  able  to  turn  the  cell  off. 
This  is  exactly  the  behaviour  caused  by  the  tangential  partitioning  of  the  operator. 

More  recently,  [Bolz  et  al.,  1989]  observed  facilitatory  interactions  between  similarly  oriented 
layer  5  and  layer  6  pyramidal  cells.  The  layer  5  cells  had  small  receptive  fields  (RF's)  which  all  lay 
within  the  RF  of  the  layer  6  cell.  By  inhibiting  the  activity  of  one  or  more  of  the  layer  5  cells,  the 
length  response  of  the  layer  6  cell  was  reduced  by  the  size  of  the  inhibited  RF.  They  concluded 
that  the  layer  6  cells  were  "constructed"  by  an  additive  summation  of  the  shorter  layer  5  cells. 
This  description  corresponds  exactly  to  the  model  described  in  this  paper. 

Potentially  problematic  is  the  work  of  [Schumer  and  Movshon,  1984],  which  discredits  the 
[Marr  and  Hildreth,  1980]  AND-gating  model,  and  can  also  be  extended  to  the  current  analysis. 
Schumer  and  Movshon  demonstrate  that  there  is  no  real  minimum  length  of  bar  stimulus 
required  to  stimulate  a  simple  cell,  instead  the  minimum  length  required  to  produce  a  noticeable 
stimulation  is  inversely  related  to  stimulus  contrast.  While  this  is  certainly  a  disproof  of  the  very 
simplistic  thresholded  AND-gating  model  of  Marr  and  Hildreth,  it  is  not  inconsistent  with  the 
model  we  are  proposing.  To  see  this,  one  must  pay  close  attention  to  the  consequences  of  the 
smooth  approximations  to  the  L/L  AND  and  OR  operators  described  in  Appendix  A.  For  any 
value  of  p  less  than  oo  (the  "hard"  AX  the  consequence  of  the  combination  of  a  zero  input  with  a 
positive  input  is  to  reduce  but  not  eliminate  the  output  (|x|  bp  0  =  |x|/2).  Thus  if  all  tangential 
regions  but  one  are  giving  zero  responses,  the  effect  will  be  to  significantly  reduce  the  response 
of  the  L/L  operator.  The  significant  observation  is  that  the  "soft"  L/L  combinators  within  the 
RF  summation  network  are  sensitive  to  changes  in  contrast-polarity  but  insensitive  to  changes  in 
contrast-magnitude.  The  analysis  of  [Schumer  and  Movshon,  1984]  serves  only  to  discredit  the  use 
of  contrast-magnitude  sensitive  logical  length  summation  models,  their  observations  are  entirely 
consistent  with  the  L/L  model. 

The  reader  may  have  noticed  that  in  Figure  5(c)  the  response  region  is  offset  from  the  actual 
peak  of  the  ID  signal  by  a  small  amount.  In  light  of  the  insistence  that  the  operator  respond 
only  at  peaks,  this  offset  may  seem  problematic  until  the  significance  of  an  observation  made  in 
[Watt  and  Morgan,  1983;  Whitaker  and  Walker,  1988]  is  recognized.  It  was  observed  that  when 
a  discrimination  task  involved  locating  the  position  of  dot  clusters  or  lines  with  non-constant 
intensity  cross-section,  the  locations  chosen  were  best  described  as  the  centroids  of  the  intensity 
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distributions.  The  behaviour  seen  with  our  model  is  consistent  with  this  interpretation,  as  the 
centroid  of  the  triangular  region  is  «  56  which  is  definitely  inside  the  region  of  positive  response. 
To  see  why  the  model  should  exhibit  this  behaviour,  notice  that  the  peak  of  a  gaussian  convolution 
with  a  positive  signal  approaches  the  centroid  of  the  signal  as  the  width  of  the  gaussian  increases. 
As  long  as  the  gaussian  is  wide  with  respect  to  the  signal,  the  peak  of  its  response  (as  determined 
by  the  peak  detection  property  of  the  line  operator)  will  be  close  to  the  centroid  of  the  intensity 
distribution.  This  relationship  should  be  investigated  further. 

A  number  of  more  specific  comparisons,  relating  our  model  to  neurophysiological  and 
psychophysical  observations  were  explored  in  [Dobbins  et  al.,  19901,  and  will  be  expanded  in 
future  work. 

5.2  Neural  Networks 

Any  system  whose  behaviour  depends  on  a  linear  summation  of  input  values  can  be  redesigned 
to  take  advantage  of  the  properties  of  the  L/L  combinators.  Candidates  other  than  the  vision 
operators  already  discussed  include  many  "neural  network"  algorithms. 

Consider,  as  a  typical  example,  the  feed-forward  networks  described  in  [Rumelhart  et  al., 
1986al.  Here  and  activation  function  F  is  applied  to  a  weighted  sum  of  inputs  o  to  generate  the 
updated  activation  a‘+1  for  each  node  i  in  the  network.  Thus 

«,'+1  =  »;)-«.)  (32) 

i 

The  weighted  sum  in  this  expression  is  just  a  one-dimensional  discrete  convolution  over  the 
previous  output  values  o‘.  Therefore,  an  L/L  decomposition  of  this  summing  network  can  lead 
to  performance  improvements  analogous  to  those  in  our  image  operators. 

We  have  already  applied  L/L  decomposition  to  a  related  relaxation  labelling  system  [Hummel 
and  Zucker,  1983;  Iverson,  19881  for  exposing  patterns  of  consistency  in  our  image  operator 
responses  (i.e.  to  suppress  noise  and  impose  a  more  specific  model  of  curve  continuity).  In 
this  particular  system,  the  addition  of  L/L  decompositions  of  the  support  network  allows  for 
successful  and  stable  convergence  of  the  relaxation  in  as  few  as  two  or  three  iterations.  The  details 
of  this  application  will  be  discussed  in  a  companion  paper. 

5.3  Conclusions 

One  of  the  major  problems  with  linear  operator  approaches  to  detecting  image  curves  is  their  false¬ 
positive  responses  to  uncharacteristic  stimuli.  After  outlining  the  necessary  logical  conditions 
for  the  existence  of  an  image  curve,  we  developed  an  operator  decomposition  which  allows  for 
the  efficient  testing  of  these  conditions,  and  the  elimination  of  these  false-positive  responses.  To 
achieve  this,  it  was  essential  to  consider  both  the  cross-section  of  the  intensity  image  and  the 
low-order  differential  structure  of  the  curve  itself. 

The  model  appears  consistent  with  both  psychophysical  and  neurophysiological  data  and 
indicates  non-linear  interactions  between  simple  cells  in  the  inter-blob  regions  of  layers  4, 5  and  6 
of  striate  cortex.  The  model  also  suggests  functional  roles  for  dendro-dendritic  interactions  and 
shunting  inhibitions  in  the  dendritic  trees  of  these  cells. 

More  generally,  we  have  introduced  a  flexible  language  for  describing  a  useful  class  of 
non-linearities  in  operators.  This  language  of  logical/linear  operators  serves  to  combine  existing 
linear  models  with  logical  descriptions  of  structure  to  produce  operators  which  have  guaranteed 
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stable  behaviour.  This  class  of  operators  represents  a  new  approach  to  the  problem  of  translating 
linear  measures  into  logical  categories.  Thus  they  may  prove  essential  in  the  eventual  solution  of 
a  wide  variety  of  classification  problems,  and  in  the  principled  and  realistic  modelling  of  neural 
networks. 
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A.  Logical/Linear  Combinators 


As  was  described  in  §2.1  a  set  of  conditionally  linear  logical  combinators  have  been  developed. 
The  logical  structure  of  these  combinators  is  based  on  the  principle  that  the  sign  of  a  signal 
(positive  or  negative)  can  be  associated  with  a  truth  value  (true  and  false,  respectively).  If  some 
logical  condition  on  the  inputs  is  found  to  hold  then  the  output  will  be  a  linear  combination  of 
the  inputs. 

The  examples  outlined  are  the  L/L  combinators  analogous  to  logical  and  and  or,  and  the 
rectifying  operators  |x|  and  4>{x).  The  notation  adopted  reflects  the  combination  of  linear  and 
logical  principles  in  these  operators: 

•  logical/linear  AND  of  x  and  y  is  x  4  y, 


•  logical/linear  OR  of  x  and  yisxl/y. 

The  definitions  and  proofs  in  §2  clarify  the  relationship  between  these  operators  and  both 
logical  and  linear  principles.  The  symmetry  and  associativity  of  these  operators  are  also  easy  to 
demonstrate. 

A  useful  formulation  of  these  operators  is  expressed  in  terms  of  the  step  function  o(x): 


|1,  ifx>0; 

1 0,  otherwise. 


(33) 


This  function  is  a  choice  operator  pivoting  around  zero,  and  as  such  it  can  be  used  to  directly 
define  the  L/L  combinators  above.  Consider  that  the  operators  can  be  defined  as: 


x  fyy  =  (x  unless  {x  >  0  A  y  <  0})  +  ( y  unless  {y  >  0  A  x  <  0})  (34a) 

x  ^  y  =  (x  unless  {x  <  0  A  y  >  0})  +  {y  unless  {y  <  0  A  x  >  0})  (34b) 

<t>(x)  =  x  when  {x  >  0}  (34c) 

|x|  =  (x  when  {x  >  0})  +  (-x  when  {x  <  0})  (34d) 


Verify  that  these  are  equivalent  to  the  algebraic  expressions 


x  4  y 

=  x(l  -  <r(x)  <r(-y))  +  y(l  -  cr(y)  ct(-x)) 

(35a) 

X  y 

=  x(l  -ff(y)<7(-x))  +  y(\-o(x)cr (-y)) 

(35b) 

1*1 

=  x<t(x)  -  x<t(-x) 

(35c) 

<t>{x) 

=  xa(x) 

(35d) 

These  are  derived  based  on  the  principle  of  selecting  those  cases  in  which  either  x  or  y  contributes 
linearly  to  the  output. 

In  order  to  derive  an  analytic  form  for  these  operators  we  adopt  a  smooth  approximation  to 


f  0,  if  x  <  0; 
l  e-1/,x,  otherwise. 


(36) 


’The  logistic'  sigmoid  function  of  {Rumeihart  et  al.,  1986b)  is  another  option  for  this,  but  the  fact  that  the  function 
chosen  is  only  non-singular  (i.e.  0  <  ap(z)  <  Don  z  €  [-1/p,  1/p)  means  that  the  "hard"  logic  of  §2.1  still  applies  for 
values  outside  of  this  region. 
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(a)  x  4P  y,  p  =  0 

(b)  x  4P  y,  p  =  2 

wm- 

(c)  x  4p  y,  p  =  8 

Wm- 

(d)  x  4p  y,  p  =  oo 

0- 

(e)  x  tip  y,  p  =  0  (0  *  tfp  v.  P  =  2  (g)  *  ^ p  y,  p  =  8  (h)  x  y,  p  =  oo 


Figure  19:  Graphs  of  L/L  combinators  varying  p.  (a),  (b),  (c),  and  (d)  show  x  4P  y, 
(e),  (f),  (g),  and  (h)  show  x  y. 


ap{x) 

This  function  is  differentiable  and 


m/2+px) 

f(l/2  +  px)  +  f(l/2-px) 


(37) 


<*(.*)  =  ton  ^(*)-  (38) 

p— *oo 

Furthermore  this  approximation  makes  clear  the  relationship  between  the  linear  sum  and  the  L/L 
operators  since  a0(x)  =  1/2.  In  this  degenerate  case,  both  of  the  operators  simplify  to  a  linear 
combination 


x  Ao  2/  =  x(l  -  <T0(x)  <ro(-y))  +  y(l  -  <*o(y)  tfo(-z)) 

(39a) 

=  3/4  (x  +  y) 

xfyjy  =  z(l  -<ro(y)<7o(-x))  +  y(l  ~  ^o(x)<T0(-y)) 

(39b) 

=  3/4  (x  +  y) 

Thus,  the  approximating  functions  and  $p  form  a  continuous  deformation  from  a  linear 
combination  to  the  absolute  L/L  operation  as  p  goes  from  0  to  oo. 

A  local  network  model  of  a  A  circuit  is  shown  in  Figure  20.  This  model  is  based  on  a  simple 
principle,  the  shunt  switch.  Such  a  switch  acts  as  a  resistive  element  for  its  input  signal  with  the 
resistance  controlled  by  the  control  input.  When  the  control  input  is  negative,  then  the  shunt  is 
on  and  has  a  very  low  resistance;  it  passes  its  input  through  effectively  unchanged.  When  the 
control  is  positive  the  shunt  is  off  and  the  resistance  is  very  high;  the  output  is  effectively  zero. 
For  simplicity,  the  shunts  pictured  allow  multiple  control  inputs,  all  of  which  must  be  positive 
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Figure  20:  Network  implementation  of  A  using  a  switching  element.  The  shunts 
(small  circles)  operate  by  passing  their  input  signal  unchanged  unless  all  of  the 
control  inputs  are  positive,  in  which  case  the  output  of  the  shunt  is  zero.* 


for  the  shunt  to  turn  off.  This  mechanism  is  similar  to  the  functional  characteristics  of  a  relay 
or  transistor.  In  neurophysiological  terms,  it  could  be  related  to  shunting  inhibition  [Fatt  and 
Katz,  1953;  Rail,  1964]  in  the  dendritic  tree  of  a  single  neuron.  The  controls  in  this  case  would  be 
realized  as  axo-dendritic  or  dendro-dendritic  synapses. 
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A  Logical/Linear  Model  of  Cortical  Subunit  Interactions. 
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1.  Motivation 

The  goal  of  the  visual  system  is  to  provide  a  description  of  the  visual  world,  an  essential 
feature  of  which  is  the  accurate  categorization  of  stimuli.  Curves  in  particular  require 
both  existence/non-existence  decisions  and  categorical  distinctions  (e.g  between  lines 
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Logical/Linear  Cell  Construction 
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Logical/Linear  Combination 


Logi$al/Unear 


4.  Standard  Experimental  Assays 

We  examine  the  response  of  the  linear  model  and  the  logical/linear  model  with  bar 
and  sinusoidal  grating  stimuli.  The  linearized  RFs  of  all  models  are  identical. 

Frequency  Response.  There  is  essentially  no  difference  in  the  magnitude  of  the 
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Grating:  Frequency  Response 
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Frequency 


Grating:  Phase  Response 
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Line:  Length  Response 
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Line  Contrast  Variation 


Interrupted  Line:  Contrast  Response 
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Interrupted  Line:  Position  Response 
(50%  Opposite  Contrast) 
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Interrupted  Line:  Orientation  Response 
(20%  Similar  Contrast) 
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range  of  the  cell.  We  propose  that  the  suppression  observed  is  not  principally  an 
orientation  effect,  but  may  be  due  to  the  expression  of  reverse  contrast  nonlinearities  in 

the  RF. 


Cross-Orientation  Inhibition 


Two  Gratings:  Orientation  Response 
(2nd  Grating  at  50%  Contrast) 


asuodsey 


Orientation 


Two  Lines:  Orientation  Response 


Orientation 


Vernier  Sensitivity 

Introduction  The  human  visual  system  is  exquisitely  sensitive  to  the  departure  from 
collinearity  (vernier  offset)  of  parallel  line  segments  (Westheimer  &  McKee,  1977).  It  has 
been  suggested  that  human  sensitivity  to  vernier  offsets  can  be  accounted  for  by  the 
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but  may  not  be  necessary  in  light  of  this  significant  enhancement  of  orientation  speci¬ 
ficity. 


Vernier  Offset 


Vernier:  Orientation  Response 
(Offset  0.0) 
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Orientation 


Vernier:  Orientation  Response 
(Offset  0.5) 


asuodsay 


Vernier:  Orientation  Response 
(Offset  1 .0) 


asuodsay 


Orientation 


7.  Supported  Personnel 


Appendix  Ml 


Endstopped  Visual  Cortical  Neurons  I:  Analysis  of  a  Quasi-Linear  Model. 
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Abstract 

A  model  of  endstopped  simple  neurons  is  mathematically  analysed  and  studied  quantitatively 
in  simulation  to  examine  its  ability  to  provide  robust  local  curvature  estimates  The  model  arises 
from  differences  in  the  response  of  cells  with  small  and  large  receptive  fields  (RFs).  Conceptually, 
the  small  RF  cell  forms  the  discharge  region  and  the  large  RF  cell  forms  the  inhibitory  end  zones 
The  combination  of  the  component  responses  is  nonlinear  although  under  some  circumstances 
an  endstopped  model  instance  with  simple  components  is  linear  The  analysis  of  this  simplified 
case  provides  insight  into  the  behaviour  of  the  significantly  more  nonlinear  versions  of  the  model 
To  study  robustness,  the  model  is  examined  in  simulation  with  curves  of  varying  radius,  width 
and  orientation.  The  sensitivity  to  perturbations  of  these  curve  parameters  depends  critically  on 
the  choice  of  model  parameters.  If  the  central  discharge  region  and  end  zones  are  matched  in 
spatial  frequency  response,  then  the  form  of  curvature  response  is  invariant  along  this  dimension 
Similarly,  matching  the  components  in  orientation-tuning  leads  to  stable  curvature  response  when 
the  curve  orientation  is  perturbed.  The  model  predicts  that  the  response  of  endstopped  neurons 
in  curvature-arc  length  space  is  dominated  by  change  in  arc  length  for  short  arcs  and  by  change 
in  curvature  for  long  arcs. 
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1  Introduction 


1.  Introduction 

Endstopped  neurons  in  the  visual  cortex  are  defined  by  their  length-selectivity,  but  also  differ 
from  nonendstopped  neurons  in  their  response  to  curved  stimuli  Hubei  and  Wiesel  (1965)  first 
identified  hypercomplex  or  endstopped  cells  and  reported  that  they  respond  well  to  stimuli  less 
than  some  length,  to  corners,  and  to  line  terminations.  They  also  provocatively  suggested  that 
endstopped  cells  should  respond  well  to  highly  curved  stimuli  and  might  in  a  sense  measure 
curvature.  We  have  provided  evidence  that  endstopped  neurons  are  curvature-selective  when 
evaluated  with  ideal,  smooth  curves,  and  have  developed  a  model  of  endstopped  simple  (ES) 
cells  which  exhibits  this  property  (Dobbins  et  al.,  1987)  The  evidence  and  model  support  a 
functional  division  in  which  endstopped  neurons  respond  to  high  curvature  and  endfree  neurons 
to  low  curvature,  each  at  their  preferred  orientation. 

In  this  paper  our  specific  motivations  are  as  follows.  First,  to  quantitatively  examine  the 
response  characteristics  of  the  quasilinear  ES  model  (linear  stages  followed  by  rectification)  and 
its  linear  approximation.  The  latter  can  be  analysed  in  a  rather  complete  way.  Second,  to  examine 
the  range  of  stimulus  conditions  under  which  curvature  response  is  qualitatively  invariant,  and 
to  examine  curvature  resolution  by  measuring  quantitative  response  variation.  Such  results  are 
indispensable  to  the  question  of  the  sufficiency  or  shortcomings  of  local  estimates  in  the  curve 
inference  process.  In  a  companion  paper  complex  components  are  introduced  and  endstopped 
model  instances  with  simple  and  complex  components  are  compared. 

The  models  considered  can  be  summarized  by  a  single  functional  form  (see  Eq.  1),  in  which 
the  response  arises  from  the  Hdifference"  of  the  responses  of  cells  with  short  and  long  receptive 
fields.  This  characteristic  can  be  implemented  by  a  local  circuit  within  an  orientation  column 
(McGuire  et  al.,  1984;  Bolz  L  Gilbert,  1986),  and  may  be  the  simplest  generic  mechanism  for 
building  complex  properties  from  simpler  ones.  In  the  endstopped  simple  (ES)  model  linear  simple 
cell  components  are  combined  nonlinearly,  while  subsequent  models  contain  one  or  more  nonlinear 
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(complex)  components  (Dobbins  et  al ..  1990).  How  the  choice  of  parameter  values  —  the  sizes, 
gains  and  receptive  field  fine  structure  —  affects  the  behaviour  of  the  model  is  evaluated  in  each 
case. 


Figure  1  About  Here 

Figure  1  illustrates  the  principal  components  of  the  model  The  mapping  from  the  input  to 
the  output  of  a  neuron  is  separated  into  two  parts:  a  spatial  integration  followed  by  a  rectifying 
function.  To  evaluate  the  behaviour  of  the  model,  we  systematically  examine  subsets  of  it,  for 
example  by  eliminating  or  simplifying  the  nonlinearities.  By  this  method  one  can  see  how  each 
element  contributes  to  the  response  specificity  of  the  model 

2.  Methods 

Simulations  were  developed  and  run  on  a  Symbolics  3670  LISP  computer.  Most  of  the 
experiments  reported  here  examine  the  response  of  the  model  to  curved  arcs  of  different  radii 
with  varied  curve  position,  orientation  or  width.  Line  and  curve  stimuli  were  either  binary-valued 
curves  with  a  jagged  appearance,  or  gray-level  curves  (8  bit)  with  a  smooth  appearance  created 
using  a  supersampling  technique.  In  this  technique  the  intensity  value  of  a  pixel  is  determined 
by  considering  the  fraction  of  coverage  of  the  pixel  (treated  as  an  8  by  8  grid)  by  a  continuous 
curve.  Line  stimuli  were  chosen  to  be  less  than  or  equal  to  the  excitatory  region  in  width,  and 
(unless  stated  otherwise)  were  centered  upon  it. 

The  model  is  formulated  in  terms  of  integration  of  continuous  variables  over  a  continous 
domain.  The  simulations,  in  contrast,  involve  summation  of  finite  precision  variables  over  a 
discrete  domain.  Simple  cell  RFs  were  represented  as  2-D  arrays  of  double  precision  floating  point 
numbers.  The  array  elements  were  the  samples  of  the  continuous  RF  lying  on  a  rectangular  grid. 
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3  The  Endstopped  Mode' 


Fine  sampling  was  employed  to  minimize  the  difference  in  numerical  results  between  rectangular- 
sampled  and  continuous  RFs.  That  is,  RFs  were  represented  by  large  arrays  e.g.  60  by  60 
elements.  Representations  of  Gaussians  were  truncated  at  ±2^ 

The  simulation  experiments  were  the  following: 

(1)  Length  Tuning.  Response  to  an  optimally-oriented  and  positioned  light  bar  is  computed 
as  length  is  added  symmetrically  to  each  end  of  the  bar. 

(2)  Orientation  Tuning.  To  evaluate  the  orientation  response  of  the  model,  a  simulated  light 
bar  is  positioned  over  the  central  RF  and  response  is  computed  at  each  orientation  as  the  bar  is 
rotated  in  equal  steps  about  its  center. 

(3)  Position  Tuning.  The  response  to  bars  or  curves  of  the  preferred  orientation  is  computed 
as  a  function  of  position  on  the  RF. 

(4)  Width  Tuning.  The  width  of  the  bar  or  curve  is  systematically  varied. 

(5)  Curvature  Tuning.  Curvature  response  is  evaluated  by  computing  the  response  to  curved 
arcs  of  varying  arc  length  and  radius  arranged  so  that  the  mid-point  of  the  arc  falls  on  the  central 
RF  and  oriented  in  such  a  way  that  the  tangent  at  mid-arc  corresponds  to  the  preferred  orientation 
of  the  RF  of  the  cell.  Except  where  stated  otherwise  the  curves  were  semi-circular  and  were  either 
thin  lines  or  edges.  In  some  experiments  the  curvature  response  is  computed  for  curves  rotated 
through  some  angle,  or  shifted  in  position  on  the  RF. 

3.  The  Endstopped  Model 

The  essential  character  of  the  endstopped  model  is  that  its  response  arises  from  the  “dif¬ 
ference"  in  response  of  cells  with  receptive  fields  (RFs)  of  similar  position  and  orientation  but 
different  size.  The  small  (5)  and  large  (L)  component  cells  and  endstopped  simple  cell  ( ES ) 
are  indicated  with  superscripts  while  subscripts  (e,  o )  refer  to  even  or  odd  RF  symmetries.  Each 
convolution  result  (Rs,  RL\  /?  «  kl  *  /)  is  passed  through  a  function  (d>(-))  which  models  the 
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4  Analysis  of  the  Endstopped  Model 
inability  of  neurons  to  represent  negative  values  on  a  low  spontaneous  firing  baseline.  Scalar 
constants  and  are  introduced  to  balance  the  responses  between  >  and  L  Therefore  the 
response  of  the  endstopped  cell  is  given  by: 

Res  =  o{cS  ■  o{RS)  -  c1  ■  o(Rl  )).  (1) 

The  activation  (or  positive  part)  function  o  will  be  used  to  clarify  the  relationship  between  certain 
linear  and  nonlinear  models.  In  this  and  previous  papers  (Dobbins  et  af ..  1987,  1989)  simple  half 
wave  rectification  is  employed  for  this  purpose  while  in  the  accompanying  paper  a  more  general 
function  is  employed  (Dobbins  et  al.,  1990). 


3.1  Simple  Receptive  Field  Components 

We  model  the  response  of  simple  neurons  as  spatial  integrations  of  the  RF  against  the  image 
and  represent  the  RFs  by  Gabor  functions  (see  Appendix  1).  This  class  of  model  has  been  shown 
to  provide  a  good  fit  to  the  spatial  RFs  of  simple  cells  (Jones  &  Palmer,  1987),  although  other 
models  also  could  have  served  as  well  (Parker  &  Hawken,  1988).  The  RF  parameter  values 
are  constrained  to  be  compatible  with  the  evidence  about  striate  cortical  simple  cells.  Since 
simple  RFs  have  a  small  number  of  parallel  zones  —  one  or  two  to  five  (Heggelund,  1986), 
and  the  neurons  do  not  respond  to  diffuse  flashes,  both  the  number  of  regions  and  the  DC 
response  component  are  limited.  Few  simple  cells  are  exactly  odd  or  even  symmetric  in  their  RF 
organization,  but  many  are  approximately  so  (Heggelund,  1986;  Jones  &  Palmer,  1987).  In  the 
simulations  described  here  only  even  (e)  or  odd  (o)  kernels  are  employed  (denoted  by  kt  and 
ko,  respectively).  Integration  is  followed  by  the  rectification  function  (<t>)  described  in  the  last 
section.  The  issues  that  arise  in  specifying  the  RF  parameters  are  treated  in  subsequent  sections. 
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4  Analysis  of  the  Endstopped  Model 


We  begin  the  analysis  of  the  (non-linear)  endstopped  model  (1)  with  a  preliminary  examination 
of  linear  RF  models  of  endfree  and  endstopped  cells.  Although  the  linear  model  is  only  partly 
successful  in  capturing  the  characteristics  of  endstopped  cells,  it  provides  the  foundation  from 
which  more  realistic  models  will  be  developed. 


4.1  Linear  RF  Models 

The  linear  RFs  are  modeled  by  Gabor  functions  (see  Appendix  1).  Low  modulation  frequencies 
result  in  RFs  like  those  shown  schematically  in  Figure  2.  The  spectrum  of  their  Fourier  transform 
is  also  shown  (a  pair  of  Gaussians  shifted  by  the  modulation  frequency  with  aspect  ratios  that  are 
the  inverse  of  those  in  the  spatial  domain).  The  simple  RFs  of  Figure  2a  (A>  (even),  kQ  (odd)) 
are  modulated  by  a  sinusoid  of  frequency  u0  in  the  x  direction.  Introducing  cosine  modulation 
in  the  y  direction  (by  v0)  yields  a  first  approximation  to  endstopped  RFs  (Figure  2b,  2c).  These 
are  denoted  (kee>koe)  where  the  second  subscript  indicates  the  additional  cosine  modulation. 


Figure  2  About  Here 


As  elaborated  in  Appendix  1,  the  location  of  the  envelopes  in  the  frequency  plane  in  Figure  2 
provides  an  indication  of  preferred  orientation  in  response  to  long  lines  and  gratings. 


Figure  3  About  Here 


Let  k  denote  a  simple  RF  kernel  and  KitsFouriertransform  : 
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k(x,y)  4=>  A'(u.r). 

The  response  to  long  lines  can  be  found  by  integrating  along  the  RF  length, 

f  lc(r,y)dy  <=>  A'(u.O).  (2) 

or  equivalently  by  examining  the  values  of  A'  on  the  it  axis.  Length  response  as  a  function  of 
v0  is  shown  in  Figure  3a.  Observe  (in  Figure  2b)  that  as  v0  increases  the  Gaussian  envelope 
K  (the  Fourier  transform  of  the  spatial  RF)  moves  off  the  it  axis  and  therefore  the  response 
to  long  lines  decreases.  Figure  3b  expresses  the  asymptotic  end  inhibition  as  a  function  of  the 
spatial  wavelength  of  modulation  (Ay  =  2ir/v0).  This  curve,  which  represents  the  strength  of 
end  inhibition  for  a  one  parameter  family  of  linear  RFs,  resembles  the  data  on  the  distribution  of 
population  end  inhibition  as  a  function  of  discharge  zone  length  reported  for  cat  striate  cortex 
(Fig.  8a  in  Kato  et  al.,  1978).  They  found  that  only  cells  with  short  excitatory  RFs  have  strong 
endstopping,  and  that  long  RFs  invariably  possess  weak  or  no  end  inhibition. 


Figure  4  About  Here 


There  are  three  principal  points  concerning  the  orientation  response  of  the  RF  types  shown 
in  Figure  2c:  (i)  the  orientation  bandwidth  for  short  stimuli  has  increased;  (ii)  the  preferred 
orientation  for  long  lines  is  shifted  toward  diagonal  orientations;  and  (iii)  the  orientation-tuning 
curve  for  long  lines  has  bifurcated.  Figure  4a  shows  that  for  a  sequence  of  RFs  (as  in  Figure 
2b)  the  orientation  response  curves  bifurcate  as  v0  increases.  Figure  4b  shows  the  orientation 
response  for  short  and  long  lines  for  a  ktt  instance.  The  response  to  long  lines  is  bimodal  while 
the  response  to  short  lines  is  single  peaked.  Measured  with  stimuli  short  enough  to  elicit  a 
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unimodal  response  curve,  one  finds  that  the  breadth  of  response  increases  with  increasing  v0 
That  is,  orientation  response  broadens  with  increased  endstopping  Kato  et  al  (1978)  report 
that  orientation  tuning  is  broader  in  endstopped  ample  and  complex  cells  than  in  their  respective 
nonendstopped  counterparts. 


Figure  5  About  Here 


Having  shown  that  linear  “endstopped”  RFs  exhibit  different  preferred  orientations  for  bng 
and  short  stimuli  we  consider  the  response  to  angled  stimuli.  By  examining  the  symmetries  of 
the  RFs  in  Figure  2c  it  follows  that  they  will  also  respond  to  chevrons.  The  k((  RF  has  four  fold 
symmetry  and  will  respond  well  to  any  of  the  four  chevrons  that  can  be  formed  from  the  ‘X’  of 
preferred  orientations.  In  contrast,  the  koe  RF  responds  well  only  to  a  chevron  of  one  contrast 
and  orientation,  and  its  mirror  in  contrast  and  orientation.  The  preferred  orientation  of  these  RFs 
can  be  approximated  as: 

9  =  =  cot~l(  —  )  (3) 

A*  u0 

Therefore  the  angular  step  (A 0)  of  the  preferred  chevron  can  be  determined  from  the  preferred 
orientation.  In  the  simulation  of  Figure  4a  the  orientation  peak  is  shifted  approximately  22  degrees 
from  vertical  (Eq.  7  predicts  21.8  degrees),  implying  the  best  chevrons  have  interior  angles  of 
either  44  or  136  degrees.  Although  a  chevron  represents  a  step  in  orientation  and  an  impulse  in 
curvature,  over  some  bcal  neighbourhood  of  arc  length  Ajj,  it  has  the  same  average  curvature 

t  • 

(«)  as  some  smooth  curve: 


7 


4  Analysis  of  the  Endstopped  Mode 


The  arc  length  (As)  in  the  chevron  curvature  approximation  is  calculated  from  \z/2  and 
Ay/2  which  are  approximations  to  local  maxima  of  an  even-symmetric  RF.  One  can  calculate  the 
peak  curvature  response  for  kee  by  finding  the  radius  of  the  circle  passing  through  three  points 
(the  center  (0,0),  and  (A* /2,  ±Xy/2)): 


k  = 


4Ag 

A^  +  Aj- 


For  smooth  curves  and  chevrons  wide  enough  to  fill  most  of  the  central  excitatory  RF,  and  of  the 
same  average  curvature,  the  linear  RF  models  respond  equally  well,  while  for  thinner  curves  there 
is  less  response  to  chevrons.  The  reason  is  that  the  smooth  curve  with  its  continuous  tangent 
provides  more  excitatory  drive  than  the  corner. 

Figure  5  illustrates  the  curvature  response  of  the  same  sequence  of  RFs.  The  first  two  (v0  = 
0,  0.3)  respond  best  to  straight  (or  low  curvature)  curves  while  the  latter  two  (r0  =  0.6,  0.9) 
exhibit  a  bandpass  curvature  response.  From  (5)  the  predicted  peak  radius  of  curvature  for  v0  - 
0.6  is  12.1  and  for  v0  =  0.9  is  6.3.  From  Figure  5a  the  peaks  occur  at  approximately  10  and  5 
respectively. 

The  advantage  of  this  class  of  model  is  clearly  its  simplicity  —  at  a  particular  RF  scale  (<rr,<xy) 
varying  one  parameter  systematically  varies  the  asymptotic  end  inhibition  and  the  preferred  curva¬ 
ture.  However  such  models  have  two  major  disadvantages.  First,  from  a  computational  perspec¬ 
tive,  since  the  highly  endstopped  instances  respond  well  to  high  curvatures  at  one  orientation  and 
low  curvatures  at  another,  to  checkerboard  patterns,  and  to  stimuli  in  the  endzones,  in  isolation 
they  can  provide  little  information  about  curvature.  Second,  RFs  with  precisely  these  properties 

—  bimodal  orientation-tuning  for  long  lines  and  excitation  flanking  inhibition  in  the  endzones 

—  have  not  been  reported.  The  second  of  these  should  perhaps  not  be  overemphasized  since 
investigators  would  ordinarily  take  pains  to  avoid  the  endzones  while  assessing  orientation  tuning. 
Given  these  shortcomings,  it  is  curious  that  the  linear  model  explains  certain  observations.  These 
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will  be  taken  up  in  the  discussion.  The  utility  of  the  linear  model  derives  from  the  fact  that 
it  can  be  completely  analysed,  and  that  nonlinear  endstopped  models  reduce  either  exactly  or 
approximately  to  a  linear  model  in  some  range  of  operation. 

4.2  A  Non-Linear  Endstopped  Model  with  Simple  Components 

If  curvature- selective  responses  are  more  than  an  artifact  of  the  responses  of  endstopped  neu¬ 
rons  then  it  is  necessary  that  the  response  to  curves  exhibit  stability  as  other  stimulus  parameters 
change.  There  are  at  least  two  possible  senses  of  stability.  A  strong  sense  is  curvature-response 
being  invariant  with  changes  of  other  curve  parameters.  A  weaker  sense  of  stability  allows  quali¬ 
tative  change  in  curvature  response  provided  it  occurs  at  a  response  level  significantly  diminished 
compared  to  the  unperturbed  stimulus.  To  be  concrete,  an  isotropic  Laplacian  of  Gaussian  RF 
exhibits  curvature  response  that  is  invariant  with  respect  to  curve  orientation,  whereas  a  differ¬ 
ence  of  oriented  operators  yields  orientation-dependent  curvature  response.  In  this  section  we 
examine  how  the  choice  of  model  parameters  affects  the  stability  of  curvature-response  for  model 
instances  in  which  both  the  small  and  large  RFs  model  cortical  simple  cells  (ES  model). 

4.2.1  The  Relationship  Between  the  Linear  and  ES  Models 

Consider  how  the  positive  part  function,  <f>  connects  linear  RF  models,  like  those  just  analysed, 
to  the  ES  model  (1)  with  linear  components.  If  the  integrations  for  the  small  and  large  components 
are  in  the  linear  range  (c*  •  r*  >  c*  •  r1,  <*  •  r5  >0),  then  the  combination  of  the  two  is  linear, 
for  certain  choices  of  the  parameter  values  the  es  model  then  closely  resembles  a  single  linear 
rf  model  (as  depicted  in  figure  2c).  in  particular,  if  the  small  (s)  and  large  (I)  simple  cells  have 
identical  spatial  frequency  (u0)  and  phase  (e,o)  of  modulation,  then  the  spatial  zeros  of  their 
rfs  correspond  and  the  difference  of  the  two  has  the  same  center  frequency  (but  not  necessarily 
bandwidth),  for  those  stimuli  which  bias  the  response  of  both  components  into  the  linear  range. 
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the  es  model  dosely  approximates  the  linear  gabor  model,  that  is.  for  appropriate  choice  of  <* 
and  J ,  a  difference  of  small  and  large  even  or  odd  simple  rfs: 

c  ■  k(  —  cl  ■  k(  ss  kff.  (6a) 

cs  •  k‘  -  d  ■  kla  ss  k0( .  (66) 

is  approximated  by  linear  endstopped  rfs  (kee ,  k0e)-  therefore,  over  some  range  of  conditions, 
such  an  es  model  will  exhibit  the  same  assets  and  liabilites  as  the  linear  models  examined  in  the 
last  section,  viewing  the  linear  endstopped  rfs  as  a  difference  of  nonendstopped  rfs  of  different 
length  illuminates  the  basis  of  curvature-selectivity  from  a  different  perspective,  having  chosen 
u®  =  ul0,  eq.  a6  implies  that  s  has  greater  orientation  bandwidth  than  I  (bic#  >  bwlff).  the 
difference  of  the  two,  Sbwg,  is  precisely  what  appears  in  the  bifurcated  orientation  curves  of 
figure  4.  hence,  since  6s  is  implicit  in  the  rf  scale,  the  average  curvature  (eq.  4)  to  which  the  rf 
is  tuned  can  be  obtained  from  the  orientation  curve  by  dividing  the  distance  between  the  peaks 
by  the  circular  arc  length,  the  breadth  of  curvature  tuning  can  also  be  approximated  from  the 
orientation  response  by  measuring  the  distance  between  the  proximal  and  distal  half  height  points 
of  the  bifurcated  response,  for  es  instances  composed  in  this  way  the  orientation-tuning  to  bng 
lines  predicts  the  curvature-response. 

at  a  particular  scale  the  nonendstopped  cells  respond  best  to  low  curvatures  while  endstopped 
cells  respond  best  to  high  curvatures,  across  scales,  however,  a  more  general  statement  is  that 
endstopped  cells  respond  to  arcs  that  span  large  orientation  changes  and  nonendstopped  cells  to 
arcs  that  span  small  orientation  changes. 

4.2.2  Varying  Parameters  Determining  Spatial  Frequency  and  Orientation  Response 
In  order  for  endstopped  neurons  to  provide  reliable  information  about  the  rate  of  curve 
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bending,  the  discharge  zones  and  the  endzones  must  satisfy  certain  constraints  These  issues  are 
explored  for  neurons  with  linear  components  in  this  section. 

Constraints  between  parameters  derive  from  the  Uncertainty  Principle  for  2D  linear  filters 
(Daugman,  1985).  The  Uncertainty  Principle  in  ID  holds  that  the  product  of  the  square  root  of 
the  variance  of  the  modulus  of  a  (normalised)  signal  in  one  domain  (A.i  =  |l«r(»'U)-«-(l))|1/J) 
and  the  transform  domain  (Au  =  (Vor(A"(u)  •  A'(u))]1,2)  is  bounded  below  by  a  definite  value. 
In  this  case  the  signal  is  the  RF  kernel.  In  two  dimensions,  two  ID  relations  hold  as  well  as  a  2D 
relation: 


AxA u  > 


4  7T 


(7«) 


AyAu  > 


4;r 


(7b) 


AiAuAyAt*  >  (7c) 

Evidently,  although  the  parameters  of  linear  filters  can  be  chosen  in  various  ways,  these 
relations  specify  limits  on  the  characteristics  that  can  be  achieved  simultaneously  in  both  domains. 
Daugman  (1985)  analyses  tradeoffs  between  orientation  and  frequency  bandwidths  which  turns 
out  to  be  significant  in  determining  the  robustness  of  curvature-response.  It  can  be  shown  that 
RFs  of  different  aze  cannot  be  simultaneously  matched  in  spatial  frequency  (peak  and  bandwidth) 
and  orientation  bandwidth.  Given  the  RF  sizes  (A yL  >  Ay5)  and  the  relation  determining 
orientation  bandwidth  (BWe  =  A* /Ay  (A6))  this  result  is  evident. 


Figure  6  About  Here 
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Figure  6  shows  different  combinations  of  small  and  large  RFs  in  the  space  and  spatial  frequency 
domain.  In  Figure  6a  the  Gaussian  envelopes  of  the  small  and  large  simple  RFs  differ  in  length 
but  not  width  and  both  are  modulated  by  a  sinusoid  of  the  same  frequency  ufl.  Therefore  in  the 
frequency  domain  they  differ  only  in  the  v  dimension.  In  Figure  6b  the  Gaussian  envelopes  have 
the  same  aspect  ratios  and  hence  differ  in  width  as  well  as  length.  Accordingly,  in  the  frequency 
domain  the  envelopes  differ  in  extent  in  both  u  and  v  dimensions.  Figure  6c  is  like  10a  except 
that  the  larger  spatial  envelope  is  modulated  by  a  lower  frequency  than  the  smaller.  Therefore, 
the  smaller  (spatial)  envelope  is  shifted  farther  from  the  origin  in  the  frequency  domain.  Figure 
6d  shows  the  case  in  which  the  small  and  large  RFs  differ  significantly  in  envelope  aspect  ratio 
and  hence  in  spatial  frequency  bandwidth  (BU’u). 


Figure  7  About  Here 


Figure  7a  and  7b  show  the  curvature-response  of  an  ES  instance  (composed  as  in  Figure 
6a)  to  curved  lines  of  different  radius  and  width  at  the  optimal  position  and  orientation.  When 
plotted  on  logarithmic  axes  it  is  dear  that  the  shape  of  the  response  does  not  change  over  a 
range  of  line  widths  (and  hence  spatial  frequencies).  Figure  7c  shows  the  response  to  curved 
lines  of  varying  curvature  and  orientation.  The  response  function  changes  both  in  magnitude  and 
shape  with  orientation.  Away  from  the  optimal  orientation  the  strongest  responses  are  to  low 
curvatures  —  stimuli  to  which  there  are  no  responses  at  the  optimal  orientation.  The  shorter 
excitatory  RF  responds  over  a  broader  range  of  orientations  to  low  curvature  stimuli  leading  to 
the  strong  response  to  oblique  lines.  As  indicated  previously  (6)  under  these  conditions  an  equal 
spatial  frequency  ES  instance  approximates  one  of  the  linear  RF  instances.  Hence  long  stimuli 
will  produce  bimodal  orientation-tuning  curves. 

An  alternative  approach  is  to  choose  the  parameters  so  that  the  orientation  bandwidth  is  the 
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Figure  8  About  Here 

same  for  the  two  components.  Equation  A6  implies  that  a  match  is  accomplished  by  satisfying: 


Ar5  At1 


and  Figure  6c  depicts  the  situation  graphically.  Figure  7d  and  7e  shows  the  curvature  response 
as  a  function  of  line  width.  Because  of  the  difference  in  component  spatial  frequency  tuning  the 
shape  of  the  curve  is  variable.  Figure  7f  shows  the  response  for  curves  of  different  orientations 
with  respect  to  the  RF.  Although  the  response  curves  vary  somewhat  in  shape,  they  decline  in 
magnitude  with  increasing  orientation  deviation  and  response  does  not  recover  strongly  at  low 
curvatures. 

Instead  of  matching  the  components  in  spatial  frequency  (</0  and  B\VU)  or  in  orientation 
(BWg)  it  is  possible  to  effect  a  tradeoff  between  the  two.  The  problem  is  how  to  choose  the 
components  so  as  to  maximize  the  invariance  of  the  curvature  response  with  changes  of  curve 
orientation  and  width.  Figure  7g,h,i  show  the  results  of  an  intermediate  choice.  Appendix  1  gives 
an  example  of  a  functional  that  one  could  minimize  in  choosing  a  tradeoff.  However  this  functional 
is  not  unique  and  different  functionals  will  lead  to  somewhat  different  parameter  choices. 

Odd-symmetric  ES  examples  are  selective  for  the  sign  of  curvature  as  well  as  its  magnitude 
(Dobbins  et  al.,  1987;  1989).  Figure  8  shows  the  response  of  an  ES  instance  in  which  both 
components  are  odd-symmetric,  but  in  which  the  parameter  values  otherwise  equal  those  in 
Figure  7g,h,i.  Essentially  only  the  magnitude  of  the  curvature  response  varies  with  curve  width 
(Figure  8a),  while  the  response  recovery  to  low  curvatures  at  oblique  orientations  is  not  that 
strong  (Figure  8b).  To  the  opposite  sign  of  curvature  there  is  almost  no  response  at  the  optimal 
orientation,  while  for  low  curvature  diagonals  the  response  approaches  the  level  of  response  shown 
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in  Figure  8b.  Overall  the  response  is  the  least  variable  observed 

The  Uncertainty  Principle  imposes  a  limitation  on  the  ability  to  match  filter  characteristics: 
the  length  difference  which  gives  rise  to  curvature- selectivity  implies  a  difference  in  Ar  for  the 
two  components.  It  is  certain  that  for  RFs  as  in  Figure  6a  (with  matched  u0  and  BWU)  that 
the  orientation  tuning  of  the  small  RF  cannot  be  increased  to  match  the  large  one.  With  linear 
components  the  position,  orientation,  and  curvature  tuning  cannot  be  decoupled. 

In  passing  we  note  that  a  corresponding  difference  in  the  length- response  of  simple  endstopped 
cells  would  be  expected  depending  on  whether  or  not  the  components  are  matched  in  spatial 
frequency.  The  implication  is  that  in  one  case  asymptotic  end  inhibition  is  independent  of  the 
width  or  spatial  frequency  of  the  stimulus  used  and  in  the  other  case  it  is  not.  An  experiment  to 
determine  whether  end-inhibition  varies  with  bar  width  is  also  of  interest  with  respect  to  the  issue 
of  whether  endstopped  and  nonendstopped  cells  exist  along  a  continuum  (Schiller  et  al.,  1976) 
or  divide  into  distinct  classes  (Kato  et  al.,  1978).  That  question  has  hinged  on  the  distribution 
of  asymptotic  end-inhibition  observed. 

Summary  The  linearity  of  the  components  of  the  ES  model  implies  that  the  Uncertainty 
Principle  establishes  a  limit  on  the  joint  space-spatial  frequency  behaviour  of  the  model.  Con¬ 
structing  endstopped  neurons  from  simple  cells  with  different  parameter  choices  leads  to  different 
tradeoffs  in  the  stability  of  curvature-tuning. 

4.2.3  Varying  Gain  and  Size 

In  this  section  we  consider  how  variation  in  the  relative  gains  and  sizes  of  the  components 
affects  curvature  response.  This  is  necessarily  preliminary  to  any  effort  aimed  at  designing  a 
family  of  mechanisms  sensitive  to  different  ranges  of  curvature. 

Since  endstopping  is  usually  defined  in  terms  of  length-tuning,  a  brief  description  of  the  effect 
of  varying  gain  and  component  size  on  length  response  is  given.  Increasing  the  gain  of  the  small 
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RF  neuron  has  the  following  effects:  response  magnitude  increases  optimal  length  increases 
(toward  the  short  RF  length),  and  the  asymptotic  response  to  long  bars  increases  (decreased  end 
inhibition).  Similarly,  increasing  the  length  of  the  small  RF  shifts  the  peak  response  to  greater 
length. 


Figure  9  About  Here 

Figure  9  shows  the  curvature  response  of  the  model  as  the  size  and  gain  of  the  small  compo¬ 
nent  is  varied.  The  left  column  shows  the  responses  in  terms  of  radius  of  curvature  and  the  right 
column  plots  the  log  of  the  response  against  the  log  of  the  radius.  The  latter  has  the  advantage 
that  the  curvature  and  radius  of  curvature  are  now  related  by  a  left-right  reversal.  Looking  down 
the  left  column  in  Figure  9  it  can  be  seen  that  the  peak  radius  increases  as  the  small  component 
increases  in  size.  Similarly,  increasing  the  gain  of  the  small  component  increases  the  peak  radius 
as  well  as  response  magnitude.  However,  the  change  in  peak  location  is  smaller  than  is  obtained 
in  length-tuning  measurements.  Hence,  over  a  limited  range,  increasing  the  size  or  gain  of  the 
small  RF  has  a  similar  effect. 

The  right  column  of  Figure  9  highlights  the  differences.  Changing  the  small  component  gain 
changes  the  rate  of  rolloff  on  the  low  curvature  side  whereas  changing  size  changes  the  response 
rolloff  rate  at  high  curvatures.  The  effect  of  changing  relative  gain  is  much  more  dramatic  than 
changing  relative  size  and  principally  determines  the  breadth  of  curvature-tuning.  For  example, 
if  one  increases  the  gain  of  the  small  RF,  the  curvature  response  width  at  half  height  increases 
and  the  end  inhibition  as  measured  by  length-tuning  decreases.  All  but  one  of  the  examples  in 
Figure  9  have  half  height  curvature  bandwidths  of  three  to  five  octaves. 

We  have  also  employed  a  smooth,  saturating  nonlinearity  with  variable  gain  and  spontaneous 
activity  in  place  of  simple  half  wave  rectification  (Dobbins  et  al.,  1990).  If  that  function  em- 
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ployed,  small  values  of  the  spontaneous  activity  parameter  can  narrow  the  curvature  bandwidth 
by  reducing  the  response  at  very  high  curvatures  On  the  other  hand,  saturation  biasing  can  sig¬ 
nificantly  broaden  the  curvature  bandwidth.  Response  is  most  stable  when  saturation  is  matched 
for  contrast  in  the  two  components.  Similarly,  for  cells  which  exhibit  contrast  gain  control  the 
minimum  change  in  the  shape  of  endstopped  curvature- response  occurs  when  the  change  in  gain 
co-varies  in  the  excitatory  and  inhibitory  components  of  the  endstopped  neuron.  Mismatch  in 
contrast  gain  adaptation  would  produce  changes  in  curvature  selectivity  as  shown  for  changing 
gain  in  Figure  9.  Changes  in  responsivity  or  noise  that  are  common  to  both  small  and  large 
components  do  not  adversely  affect  ES  model  performance,  but  uncorrelated  changes  will  cause 
perturbation  in  curvature-tuning  curves. 

The  examples  shown  in  Figure  9  have  components  matched  in  spatial  frequency  response 
and  the  results  were  for  curves  of  only  one  width.  If  the  components  are  not  matched  in  spatial 
frequency  response  then  the  way  in  which  curvature-response  varies  with  relative  gain  depends 
on  the  curve  width. 

Summary  Both  the  relative  gain  and  the  relative  size  of  the  endstopped  model  components 
affect  the  shape  of  the  curvature  response.  Relative  gain  is  more  significant  and  determines  the 
curvature  bandwidth  whereas  the  relative  size  is  more  important  in  determining  the  peak  of  the 
response  curve. 

4.3  Curvature  as  Orientation  Change 

Consider  a  curve  described  locally  as  curvature  parametrized  by  arc  length  (a  =  a(s)).  The 
orientation  range  (A0)  spanned  by  a  section  of  arc  is  the  integral  of  the  curvature  with  respect  to 
the  arc.  The  distinction  in  the  curvature  preferences  between  endstopped  and  endfree  cells  can 
be  described  in  terms  of  orientation  range.  For  arcs  at  least  as  long  as  the  maximal  RF  dimension 
endstopped  neurons  prefer  large  orientation  ranges  (e  g.  A0  >  45°)  and  nonendstopped  cells 
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small  ones.  The  orientation  range  distinction  is  scale-independent  and  hence  more  general  than 
the  low/high  curvature  distinction. 


Figure  10  About  Here 

Figure  10  shows  the  results  of  experiments  with  an  endstopped  simple  model  cell  in  which 
both  curvature  and  arc  length  are  systematically  varied.  Figure  10a  shows  the  response  surface 
in  radius  of  curvature-arc  length  space.  Figure  10b  plots  level  sets  (iso-response  contours)  in  the 
same  coordinates.  For  short  arcs,  the  response  gradient  is  in  the  direction  of  changing  arc  length, 
while  for  long  arcs  the  gradient  is  in  the  curvature  direction. 

/ 

5.  Discussion 

5.1  The  Range  of  Curvature  Selective  Cells 

Wilson  (1985)  has  argued  that  the  effective  orientation  range  employed  in  human  curvature 
discrimination  is  primarily  determined  by  the  tuning  of  the  underlying  orientation  selective  mech¬ 
anisms.  If  endstopped  cells  are  concerned  with  representing  the  local  bending  of  arcs,  we  now 
ask  how  many  (with  distinctly  different  stimulus  selectivity)  are  required. 

There  are  two  factors  to  consider:  (i).  the  range  of  curvature  that  is  represented  by  single 
RFs,  and  (ii).  the  curvature  resolution  (inverse  of  bandwidth)  attained  by  the  RFs.  There  are 
two  constraints  affecting  the  range  of  representation  by  RFs.  The  first  is  a  maximum  curvature 
constraint  which  is  ultimately  determined  by  optical  resolution  and  photoreceptor  density.  Then 
the  maximum  curvature  to  which  the  system  is  sensitive  is  determined  by  the  smallest  orientation 
selective  units.  In  the  primate,  a  substantial  fraction  of  foveal  cortical  units  is  less  than  10  arc 
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minutes  in  length  and  the  smallest  are  less  than  5  arc  minutes  (Parker  &  Hawken,  1988).  In  cat 
striate  cortex,  the  neurons  with  the  shortest  RFs  are  almost  invariably  endstopped  (Kato  et  al., 
1978),  and  ance  there  is  substantial  or  total  overlap  between  RF  center  and  endzones  (Orban 
et  al.,  1979b),  the  excitatory  RF  length  of  endstopped  cells  is  systematically  underestimated 
(Dobbins  et  al.,  1989).  Experiments  with  the  ES  model  show  that  the  peak  of  the  length 
response  curve  may  be  as  little  as  50-60  percent  of  the  length  of  the  short  excitatory  RF.  In 
addition,  we  have  conducted  other  simulations  to  determine  the  relationship  between  the  peaks 
of  the  length  and  radius  of  curvature  tuning  curves.  The  results  indicate  that  the  peak  radius  of 
curvature  varies  between  the  length-tuning  peak  (typically  0.6  -  0.75  of  the  small  RF  length)  to 
an  upper  value  equal  to  the  small  RF  length.  Therefore,  if  one  takes  a  value  of  5  arc  minutes 
as  the  smallest  measured  length,  assuming  the  measured  length  is  0.6  -  0.75  the  length  of  the 
discharge  region,  and  using  the  just- described  relation  between  length  and  radius  of  curvature 
one  obtains  an  estimate  of  peak  radius  of  5  -  9  arc  minutes  for  the  smallest  units. 

Parker  and  Hawken  (1988)  estimated  RF  lengths  indirectly  from  spatial  frequency  and  orien¬ 
tation  bandwidth  measurements  in  monkey.  Two  thirds  of  the  RFs  are  within  a  factor  of  five  in 
length  although  the  full  range  was  about  a  decade  (less  than  5  to  greater  than  40  arc  minutes). 
The  data  of  Kato  et  al.  (1978)  and  of  Gilbert  (1977)  also  indicate  that  the  range  of  RF  lengths 
at  a  particular  eccentricity  in  cat  Area  17  is  no  more  than  a  decade.  However  since  these  data  in 
fact  represent  a  range  of  eccentricites  of  a  few  central  degrees,  and  there  is  greater  variability  in 
length  measurements  than  for  many  other  RF  parameters,  it  is  possible  that  the  range  is  rather 
less. 

There  is  also  a  kind  of  minimum  curvature  constraint.  Although  it  is  possible  to  choose  RF 
parameters  to  obtain  endstopped  RFs  responsive  to  low  orientation  ranges  (but  still  curvature 
bandpass),  a  proportionally  denser  sampling  of  orientation  is  implied,  making  it  more  economical 
to  use  endfree  (curvature  lowpass)  neurons  to  represent  bw  orientation  ranges. 
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Curvature  resolution  is  not  primarily  determined  by  the  curvature  bandwidth  to  a  particular 
idealized  stimulus  but  rather  by  the  variation  in  bandwidth  over  the  class  of  perturbed  curves 
Proceeding  naively,  by  combining  adjacent  elements  in  a  series  of  four  octave-spaced  RFs  one 
obtains  three  endstopped  cells.  The  four  cells  span  a  factor  of  eight  in  length  The  endstopped 
cells  composed  in  this  way  have  constant  log  bandwidth  and  cover  a  factor  of  four  in  their 
curvature  tuning  peaks.  First  note  that  the  model  predicts  half  height  curvature  bandwidths  of 
3-5  octaves.  This  breadth  suggests  that  octave  RF  spacing  represents  significant  ^ersampling 
of  curvature.  In  other  words,  such  sampling  does  not  partition  curvature  into  orthogonal  classes. 
Second,  the  shortest  and  longest  RFs  (at  any  retinal  eccentricity)  are  very  unlikely  to  be  tuned  to 
the  same  spatial  frequency,  implying  that,  for  a  given  eccentricity  and  preferred  spatial  frequency 
there  do  not  exist  endstopped  cells  tuned  for  curvatures  differing  by  as  much  as  a  factor  of  four. 
The  qualification  to  be  made  is  that  we  are  referring  to  cells  that  are  strongly  endstopped;  cells 
with  weak  end  inhibition  extend  the  range  of  peak  curvatures  but  have  such  broad  curvature¬ 
tuning  that  it  is  unlikely  that  they  play  a  significant  role. 

To  summarize,  examination  of  the  range  of  RFs  available  for  synthesizing  endstopped  neurons 
suggests  that  the  range  of  peak  curvatures  represented  by  endstopped  cells  is  not  much  greater 
than  a  factor  of  two  (for  a  given  preferred  spatial  frequency).  Within  this  range  there  may  be 
cells  with  a  range  of  curvature-tuning  characteristics.  However,  these  would  not  appear  to  fall 
neatly  into  orthogonal  curvature  classes. 

5.2  Linear  and  Quasi-Linear  Endstopped  Models 

The  endstopped  simple  model  exhibits  the  right  qualitative  length,  orientation  and  curvature 
response.  As  illustrated  in  Figure  3b  plotting  Ay  (fraction  of  the  total  RF  that  is  excitatory) 
against  end  inhibition  leads  to  a  distribution  very  much  like  the  empirical  one  found  in  Kato  et 
al.  (1978).  If  the  spatial  extent  of  the  afferent  pool  that  may  contribute  to  RF  synthesis  is 
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fixed,  then  the  linear  model  predicts  that  endstopped  cells  have  broader  orientation  tuning  than 
their  nonendstopped  counterparts,  and  that  the  degree  of  endstopping  is  intimately  linked  to  the 
length  of  the  excitatory  RF  Both  these  results  are  in  accord  with  the  experimental  findings  of 
Kato  et  al.  (1978). 

Although  the  linear  and  quasi-linear  ES  models  exhibit  the  right  qualitative  behaviour  they  can 
be  discounted  as  plausible  endstopped  neuron  models  on  empirical  grounds:  the  phase-specific 
nature  of  the  endzones.  Tanaka  et  al.  (1987)  have  examined  endstopped  cells  in  cat  Area  19, 
finding  that  the  endzones  are  tuned  to  the  same  spatial  frequency  as  the  RF  center  but  have 
broader  bandwidth.  K  the  endzones  were  synthesized  from  one  simple  cell  this  would  imply  that 
the  endzones  are  narrower  than  the  center.  This  is  incompatible  with  the  evidence  of  Orban 
et  al.  (1979b)  on  endzone  width  in  cat  striate  endstopped  neurons  In  addition,  Tanaka  et  al. 
(1987)  found  that  the  endzones  of  the  Area  19  cells  were  phase-insensitive  implying  a  complex-like 
character  as  first  suggested  by  Hubei  and  Wiesel  (1965).  In  the  accompanying  paper  we  explore 
models  of  endstopped  neurons  with  complex  components.  In  addition  to  nonlinear  components 
one  can  imagine  nonlinear  combination  of  the  components.  For  example  one  could  treat  the  end 
inhibition  as  divisive  rather  than  subtractive.  This  AND-NOT  combination  (Koch  et  al.,  1983) 
can  be  interpreted  as  an  assertion  of  the  form:  the  contour  can  be  approximated  by  a  line  segment 
of  length  L  but  not  of  length  2 L.  Employing  divisive  rather  than  subtractive  inhibition  changes 
the  shape  of  the  length  and  curvature  response  curves  somewhat  but  not  the  basic  character  of 
the  model. 

Both  the  linear  and  quasi-linear  ES  models  respond  to  chevrons,  crosses  and  so  forth  in 
addition  to  smooth  curves.  The  RF  symmetries  imply  that  these  models  respond  to  two  differ¬ 
ently  oriented  chevrons  and  hence  smooth  curves  at  orthogonal  orientations  and  with  distinct 
curvatures.  This  undesirable  artifact  is  partly  a  consequence  of  the  position  or  phase-specificity 
of  end  inhibition.  If,  instead,  inhibition  is  position-independent  in  the  end  zones  as  the  evidence 
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suggests  then  the  response  to  curves  and  chevrons  rotated  90  degrees  with  respect  to  the  center 
is  abolished. 

All  of  the  versions  of  the  endstopped  model  (in  this  and  the  accompanying  paper)  respond 
to  curves,  none  responds  exclusively  to  curves,  and  none  provides  a  precise  estimate  of  curvature. 
Therefore  the  endstopped  simple  model  is  not  a  ‘curvature  detector  Nevertheless,  as  shall  be 
discussed  in  the  following  paper  the  curvature-selective  response  of  endstopped  neurons  can  be 
exploited  as  a  first  stage  in  cur  e  inference.  The  degree  of  success  of  the  linear  and  quasi- 
linear  models  is  attributable,  we  believe,  to  more  realistic  non-linear  models  resembling  the  linear 
model  in  some  range  of  operation.  This  can  arise  from  nonlinear  components  or  more  nonlinear 
interactions  between  them  or  both  (Iverson  and  Zucker,  1990;  Dobbins  et  at.  1990). 
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7.  Appendix  1. 

The  formulation  of  Gabor  (1946)  was  introduced  to  vision  by  Marcelja  (1980)  and  Daugman 
(1980)  and  is  used  throughout  the  paper  in  representing  linear  RF  components  because  of  its 
simplicity  and  convenience  in  both  spatial  and  spatial  frequency  domains.  Recall  that  Gabor 
functions  are  the  product  of  Gaussians  with  sinusoids.  Therefore  in  general  a  2D  Gabor  RF  can 
be  described  as: 
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where  (x0i,x02)  gives  the  center  of  the  Gaussian  envelope,  ctx12  and  er^2  the  variances  and 
p  the  covariance  of  the  Gaussian.  The  complex  modulation  frequencies  are  given  by  uQ  and  v0 
and  the  phases  by  a  and  /?.  Employing  a  loca>  coordinate  system  {.r.y)  centered  on  the  RF  and 
oriented  so  that  the  modulation  is  in  the  x  direction,  we  consider  the  following  Gabor  functions: 
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which  have  purely  even  and  odd  phase  modulation  although  the  evidence  indicates  that  simple 
cortical  RFs  exist  with  all  intermediate  phases  as  well  (Heggelund,  1986).  We  assume  that 
<Ty  >  ox  and  that  the  modulation  frequencies  are  low.  In  particular  u0  is  low  enough  that  the 
number  of  lobes  in  the  RF  is  less  than  or  equal  to  five  (4(7;  <  2.5A,).  The  spectrum  of  the 
Fourier  transform  is  also  shown  and  is  a  pair  of  Gaussians  shifted  by  the  modulation  frequency 
u0,  with  aspect  ratios  that  are  the  inverse  of  those  in  the  spatial  domain. 

Allowing  cosine  modulation  in  the  y-direction  one  obtains  something  like  endstopped  RFs: 
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fcee(I-!/;ffr-(7y;uO’l'o)  =  ke(x.y)cos{v0!i)  (43a) 

koe{x,y\Ox'Gyi^-0'Vo')  —  ko(x .  y)  cos(i  oil)  (43  b) 

Note  that  a  trigonometric  identity  allows  one  to  treat  each  of  these  doubly- modulated  RFs  as 
sums  of  RFs  with  ID  modulation.  Since  there  is  no  evidence  of  more  than  three  zones  along  the 
RF  length  the  spatial  wavelength  of  modulation  (Ay  =  2r/i0)  is  constrained  to  be  less  than  or 
equal  to  the  RF  length  (Ay  <  4ay).  In  Figure  2b  v0  is  increased  from  0  and  the  spatial  envelope 
takes  on  the  shape  shown.  The  spectrum  of  the  Fourier  transform  is  represented  schematically  on 
the  right.  It  is  worth  noting  that  spatial  frequency  has  usually  had  a  one  dimensional  interpretation 
in  neurophysiology  and  psychophysics.  For  rectilinear  stimuli,  this  is  satisfactory,  but  disks  and 
rings  have  a  radial  organization  of  spatial  frequency  (with  Bessel  kernel;  Bracewell,  1965).  In 
referring  to  spatial  frequency  bandwidth  ( BWU )  we  shall  mean  the  bandwidth  in  the  direction  of 
the  modulation  vector  (u0). 

The  Fourier  transform  of  a  long  thin  vertical  bar  has  a  spectrum  concentrated  along  the 
u-axis.  A  form  which  is  rotated  in  space  has  its  Fourier  transform  rotated  through  the  same 
angle.  In  symbols: 


/(x,  y)  =  f(r,0)  o  F(u,  u)  =  F{p .  * ).  (44) 

where  (r,0)  and  (p,-r)  are  polar  representations  in  the  space  and  spatial  frequency  domains, 
respectively.  Rotating  the  line  through  a: 

/(r,  0  +  a)  F(p ,  7  +  <*)  (45) 

Therefore  the  Fourier  transform  of  a  rotated  line  is  the  rotated  transform  of  the  original  line.  A 
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vertical  sinusoidal  grating  transforms  to  a  pair  of  impulses  on  the  «  axis  and  rotating  the  grating 
sweeps  out  a  circle  of  radius  u0  in  the  u-v  plane  The  preferred  orientation  of  an  RF  is  normal 
to  the  direction  f  the  modulation  vector,  hence  the  location  of  the  envelopes  in  the  spatial 
frequency  plane  in  Figure  2 a  and  2c  provides  an  indication  of  preferred  orientation  in  response  to 
lines  and  gratings.  The  orientation  bandwidth  of  a  simple  cell  viewed  in  the  frequency  domain 
can  be  treated  as: 


B\Ve  =  tan-l(bv/u0)  (>46) 

where  At’  is  the  variance  of  the  RF  in  v  (Kulikowski  et  a/,  1982).  or,  approximated  graphically  by 
the  arcs  from  the  origin  to  the  RF  envelope  as  shown  in  Figures  6a  and  6c  This  relation  can  be 
expressed  equally  well  in  terms  of  the  corresponding  spatial  domain  variables  (l/A,  and  1/Ay). 

Having  obtained  a  measure  of  orientation  bandwidth  we  derive  a  measure  of  the  joint  spatial 
frequency-orientation  match  of  two  simple  RPs  of  different  size.  The  degree  of  match  in  spatial 
frequency  of  the  simple  components  can  be  represented  by  the  overlap  of  the  two  Gaussian  kernels 
in  the  frequency  domain.  Therefore  the  component  matches  vary  with  the  ratio: 

)Au5/u£  -  Af^/up  | 

/  /  K  S(u,  v )  •  I\L(u,  r)  dndr 

where  the  integration  is  taken  over  the  plane  (C2  is  reduced  to  R2  for  equal  phase  RFs)  and  we 
desire  small  values  as  the  component  parameters  u0s,  u0L-  Ah$.  and  A»£  vary. 
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8.  Figure  Legends 

Figure  1.  a.  The  endstopped  model  is  constructed  from  the  convergence  of  excitation  from 
a  short  RF  cell  and  inhibition  from  a  tong  RF  cell. 

Figure  2.  a.  Even  and  odd  (ke,k0)  Gabor  representations  of  simple  RFs  in  space  (left)  and 
their  Fourier  spectrum  (right).  The  ellipses  have  Gaussian  weighting  in  both  domains,  b.  The 

transition  between  the  RFs  in  a.  and  c.  as  v0  is  increased  from  0  in  both  domains.  The  arcs 

/ 

bounding  the  envelopes  in  the  frequency  domain  provide  a  graphical  indication  of  the  orientation 
bandwidth  in  response  to  very  long  lines.  Similarly,  the  migration  of  the  envelopes  away  from 
the  u  axis  is  related  to  the  change  in  asymptotic  end  inhibition  measured  with  long  lines  (or 
spatial  DC  response),  c.  Simulating  endstopped  RFs  with  additional  cosine  modulation  in  the 
y-direction  (£„,£<*).  Note  that  in  response  to  long  bars,  the  preferred  orientation  has  shifted 
and  the  orientation  bandwidth  has  increased. 

Figure  3.  a.  Length- response  curves  for  the  fcee  Gabor  RF  for  different  values  of  v0  (0. 
0.3,  0.6,  0.9).  (RF  parameters:  Size:  21  pixels,  Aspect  Ratio:  <rylot  =  2,  u0  =  0.15)  As 
the  inhibitory  end  zones  (modulation  frequency)  increase,  the  asymptotic  inhibition  goes  from  0 
to  100  percent,  b.  The  asymptotic  end  inhibition  as  a  function  of  the  relative  lengths  of  the 
excitatory  and  inhibitory  zones.  The  abscissa  is  the  wavelength  of  the  modulating  sinusoid  (A) 
normalized  by  the  RF  length  (S). 

Figure  4.  a.  The  orientation  response  for  kee  instances  with  the  same  parameter  values  as  the 
previous  figure.  The  response  curve  bifurcates  as  v0  increases,  b.  Orientation  response  for  short 
(solid  curve)  and  long  bars  (dotted  curve)  of  width  3  pixels  for  the  instance  with  v0  =  0.06.  The 
short  bars  equal  the  excitatory  center  in  length  and  the  long  bars  the  total  RF  length.  Although 
unimodal,  the  response  to  short  bars  has  broader  bandwidth  than  the  for  the  nonendstopped 
instance  in  response  to  long  bars. 

Figure  5.  a.  The  response  to  appropriately  oriented  semi-circular  arcs  for  the  same  set  of 
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parameter  values  as  previously.  For  low  values  of  v0  the  curves  appear  nonendstopped  but  become 
increasingly  endstopped  as  v0  increases,  b.  The  data  are  replotted  on  log-log  axes.  The  curves 
differ  in  their  characteristics  both  on  the  low  and  high  curvature  side.  Plotted  this  way  curvature 
and  radius  of  curvature  are  merely  related  by  a  sign  change. 

Figure  6.  The  RFs  and  Fourier  spectra  of  different  pairs  of  vertically-oriented  simple  Gabor 
RFs  that  differ  in  length  (after  Daugman,  1985).  a.  Same  bandwidth  in  u  ( BWU )  and  center 

frequency  ( u0 )  b.  Equal  envelope  aspect  ratio  (AR)  and  u0  c.  Equal  BWU  and  orientation 

bandwidth  ( BWg ).  d.  Equal  center  frequency  u0. 

Figure  7.  The  curvature  response  as  a  function  of  curve  width  and  orientation  for  three 
ES  instances.  The  left  and  middle  columns  show  response  to  curves  of  different  width  plotted 
on  linear  and  log  axes,  respectively.  The  line  widths  are  1,  3,  5,  and  7  pixels  (solid,  dotted, 

dashed,  long-dashed).  The  right  column  shows  the  response  to  curves  of  width  3  pixels  at 

orientations  from  optimal  to  25  degrees  from  optimal  in  steps  of  5  degrees.  In  all  cases  the 
radius  of  curvature  varies  from  0  to  200  in  steps  of  10.  the  top  row  (a.,  b.(  c.)  corresponds  to 
Figure  6a  (parameters:  Size-S/L:  35/61,  Aspect  Ratio  (AR)  S/L:l. 15/2.0,  center  frequency  tig- 
S/L0.05/0.05,  and  gains  /cL: :1. 8/1.0)  designed  so  that  the  components  are  matched  in  center 
spatial  frequency  (u*>)  and  bandwidth  (BWU).  a.  The  response  as  a  function  of  curve  width, 
b.  On  log-log  axes  the  curves  appear  as  vertically  translated  copies,  c.  There  is  strong  response 
to  high  curvatures  at  the  vertical  orientation  and  low  curvatures  at  oblique  orientations.  The 
middle  row  (d„  e.,  f.)  corresponds  to  Figure  6c  (parameters:  Size-S/L.  35/61,  AR-S/L:1. 15/2.0. 
uo-S/l:0.08/0.046,  c?/cL:l. 9/1.0)  designed  so  that  the  components  are  matched  in  spatial 
frequency  ( BWU )  and  orientation  {BWg)  bandwidth.  d.t  e.  The  response  as  a  function  of  curve 
width  is  variable  and  there  is  no  response  to  the  widest  curve,  f.  The  response  narrows  and 
decays  as  the  curve  orientation  moves  away  from  vertical.  There  is  no  response  to  low  curvature 
at  oblique  orientations.  The  bottom  row  ( g h.,  i.)  represents  one  choice  of  tradeoffs  between  the 
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matching  in  spatial  frequency  and  in  orientation  (parameters  Size-S/L:  35/61,  AR-S/L:1. 15/2.0. 
uo-S/L:0. 06/0. 046,  /cL  :1. 85/1.0).  g.,  h.  Comparing  with  the  panels  above  it  is  dear  that 
the  form  of  the  response  is  less  sensitive  to  curve  width  than  in  d  and  e  but  more  so  than  in  a 
and  b.  i.  The  reverse  is  true  for  sensitivity  to  curve  orientation. 

Figure  8.  The  curvature  response  as  a  function  of  curve  width  and  orientation  for  an  odd- 
symmetric  ES  instance.  The  left  and  middle  graphs  show  response  to  curves  of  different  width 
plotted  on  linear  and  log  axes,  respectively.  The  line  widths  are  1,  3.  5,  and  7  pixels  (solid, 
dotted,  dashed,  long-dashed).  The  right  graph  shows  the  response  to  curves  of  width  3  pixels 
at  orientations  from  optimal  to  25  degrees  from  optimal  in  steps  of  5  degrees  In  all  cases  the 
radius  of  curvature  varies  from  0  to  200  in  steps  of  10. 

Figure  9.  The  curvature  response  of  three  ES  instances  as  the  gain  of  the  small  component 
is  changed.  The  right  column  re-expresses  the  data  of  the  left  column  on  log-log  (base  2)  plots. 
In  each  case  51  curves  of  radius  from  4  to  204  pixels  and  width  3  pixels  were  used.  In  a.,  b„ 
and  c.  the  small  components  are  of  length  21,  31  and  41  pixels,  respectively,  with  AR  1.0,  1.4, 
and  1.6.  In  each  case,  for  both  small  and  large  components  u0  =  0.05.  The  large  component  is 
always  the  same  -  61  pixels  long  with  AR  =  2.0,  and  =  1.0.  a.  For  the  21  pixel  RF  c5  takes 
values  2.5,  3.0  and  3.7  (solid,  dotted,  dashed)  for  the  three  curves  shown,  b.  a.  For  the  31  pixel 
RF  takes  values  1.8,  2.1,  and  2.4  for  the  three  curves  shown,  c.  For  the  41  pixel  RF  c5  takes 
values  1.3,  1.6  and  2.0  for  the  three  curves  shown.  In  each  case  the  relative  gain  determines  the 
rolloff  on  the  right  (low  curvature)  side.  The  relative  size  determines  the  rolloff  on  the  left  (high 
curvature)  side  and  the  peak  position.  The  locus  of  the  peak  radius  is  approximately  equal  to 
the  length  of  the  small  RF. 

Figure  10.  The  response  of  an  endstopped  simple  instance  to  curves  in  which  arc  length  and 
radius  of  curvature  are  systematically  varied.  The  parameters  are  as  in  Figure  11a, b,c.  a.  Three 
dimensional  response  plot.  The  termination  of  the  surface  from  the  left  foreground  shows  the 


30 


8  Figure  Legends 


curvature  response  curve  while  a  view  from  the  nght  shows  the  (reversed)  length-tuning  curve 
b.  Level  sets  of  the  response  surface.  For  short  arc  lengths  there  is  low  curvature  selectivity  and 
high  length  selectivity,  while  for  long  arcs  the  reverse  is  true  This  is  evident  by  examining  the 
direction  of  the  response  gradient  (normal  to  the  level  set  contours) 
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Endstopped  Visual  Cortical  Neurons  II. 

Experiments  with  a  Complex.  Nonlinear 
Model. 

Allan  Dobbins  Steven  W  Zucker  Max  $  Cynader 

Abstract 

A  model  of  complex  endstopped  neurons  is  developed  and  analysed  The  endstopped  neuron 
model  combines  an  excitatory  center  which  may  be  simple  or  complex  and  complex  inhibitory  end 
zones.  Complex  cells  are  classified  as  standard  or  special  according  to  positional,  length-summing 
and  orientation-tuning  criteria.  The  simulations  reported  here  suggest  there  may  be  two  factors 
contributing  to  the  special  complex  behaviour:  (i)  subadditive  synaptic  interactions  among  inputs 
and  (11)  the  presence  of  endstopping.  The  curvature  selectivity  of  the  endstopped  complex  model 
is  evaluated  with  curved  stimuli  that  are  varied  in  orientation  and  width  to  examine  the  robustness 
of  response  to  stimulus  perturbation.  The  results  depend  strongly  on  how  the  parameters  of  the 
constituents  of  the  model  cell  are  chosen.  In  some  cases  one  obtains  curvature- selective  responses 
very  similar  to  those  obtained  with  the  endstopped  simple  neuron  model  In  some  other  cases  the 
behaviour  is  more  complicated.  Finally,  we  consider  the  range  of  curvature  spanned  by  endstopped 
cells  and  discuss  representational  issues  in  curve  description. 
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1.  Introduction 

Descriptions  of  visual  objects  are  related  to  descriptions  of  their  surfaces  and  bounding  curves 
The  processes  underlying  these  share  important  features  In  neither  case  can  continuity,  linearity 
smoothness,  or  freedom  from  occlusions  be  blithely  assumed  Instead  one  needs  mformatior 
from  the  visual  image  concerning  whether  and  where  these  are  true  To  estimate  the  structure  of 
curves  one  must  answer:  Is  it  a  curve  (a  continuous  line  or  edge):  Is  it  straight  or  curved;  Does 
it  curve  smoothly,  and,  if  so,  how  rapidly?  It  follows  that  a  necessary  prerequisite  for  a  theory 
of  curve  description  is  determination  of  which  of  these  questions  can  be  addressed  locally,  and 
which  require  interactions  across  distance.  The  present  study  maintains  a  narrower  focus,  wherein 
we  continue  exploring  the  hypothesis  that  endstopping  is  a  property  related  to  the  estimation  of 
curvature,  and  evaluate  models  of  endstopped  neurons  and  their  success  in  estimating  curvature 

Dreher  (1972)  established  that  the  discharge  region  of  endstopped  cells  may  be  either  simple 
or  complex,  but  a  variety  of  evidence  implies  that  the  endzones  have  a  complex  character  (Hubei 
and  Wiesel.  1965;  Orban  et  al.,  1979a,  1979b;  Tanaka  et  al..  1987).  Therefore  in  this  paper 
we  extend  the  endstopped  simple  model  (Dobbins  et  al.,  1987;  1989;  1990)  to  include  complex 
components.  The  complex  model  parameters  can  be  chosen  to  resemble  a  cell  of  either  the 
standard  or  special  complex  type.  Length,  orientation,  and  positional  sensitivities  of  the  complex 
cell  model  are  computed  to  establish  a  baseline  of  comparison  for  simple  cells  and  the  experimental 
data.  Simulations  are  then  performed  in  which  the  curvature- selectivity  of  the  endstopped  complex 
model  is  evaluated. 

2.  Methods 

Simulations  were  developed  and  run  on  a  Symbolics  3670  LISP  computer.  Most  of  the 
experiments  reported  here  examine  the  response  of  the  model  to  curved  arcs  of  different  radii 
with  varied  curve  position,  orientation  or  width.  A  complete  description  of  the  simulation  methods 
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Figure  2  About  Here 

is  contained  in  Dobbins  et  al.  (1990) 

3.  The  Endstopped  Model 


Figure  1  About  Here 

The  essential  character  of  the  endstopped  model  is  that  its  response  arises  from  the  "dif¬ 
ference”  in  response  of  cells  with  receptive  fields  (RFs)  of  similar  position  and  orientation  but 
different  size.  The  small  (S)  and  large  ( L )  cells  can  be  simple  or  complex  (Figure  1)  Notation 
superscripts  (5,  L,  ES)  refer  to  particular  cells  while  subscripts  (i  <>)  refer  to  even  or  odd  RF 
symmetries.  Each  convolution  result  (RS ,  RL ,  Rl  =  k'  *  /)  is  passed  through  a  function  (0()) 
which  models  the  inability  of  neurons  to  represent  negative  values  on  a  low  spontaneous  firing 
baseline.  Scalar  constants  c?  and  are  introduced  to  balance  the  responses  between  .S’  and  L 
Therefore  the  response  of  the  endstopped  cell  is  given  by: 

RES  =  <t>(cS -<t>{RS)-cL  <t>(RL)).  (1) 

The  activation  (or  positive  part)  function  0  will  be  used  to  clarify  the  relationship  between 
certain  linear  and  nonlinear  models.  Previously,  simple  half  wave  rectification  was  employed  for 
this  purpose  (Dobbins  et  al.,  1987;  1989). 

3.1  The  Activation  Function 
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Observe  that  <t>  depends  on  two  parameters  T  and  o  where  r  determines  the  form  of  the 
nonlinearity  and  p  controls  the  contrast  sensitivity  <t>  contains  a  saturating  nonlinearity  to  reflect 
the  dynamic  range  limits  of  cortical  neurons  and  T  models  spontaneous  activity  which  determines 
the  degree  of  rectification  (or.  equivalently,  the  amount  of  compression  of  negative  values).  We 
have  modelled  <t>  as: 


HR.p.n  = 


1  -  e~R  '' 


(2) 


i  +  i/r  t~R 

where  R  is  the  convolution  result,  and  T  e  (0.1)  (see  Appendix  1  for  further  details  concerning 
<t>).  Observe  in  Figure  2,  that  for  small  T  the  output  has  a  saturating  half  wave  rectified  character 
while  for  T  =  1  there  is  no  rectification  and  the  range  of  output  is  equal  for  positive  and  negative 
R.  In  addition,  the  width  of  the  linear  range  is  roughly  independent  of  I"  Therefore,  apart  from 
quadratic  distortion  at  low  R  and  saturation  at  high  R,  <J>  with  small  T  approaches  simple  half 
wave  rectification  (<f>(  )): 


Sm  0  {j  (3) 

Varying  the  relative  values  of  T  and  4>  in  relation  to  the  input  magnitude  allows  selection  of 
a  variety  of  different  input-output  properties.  In  much  of  what  follows  it  is  assumed  that  each 
of  the  model  components  has  length  summation,  however  we  shall  also  consider  special  complex 
cells  which  do  not  exhibit  length  summation  and  which  are  often  endstopped. 

3.2  The  Receptive  Field  Components 

The  endstopped  cell  discharge  region  and  endzones  may  be  either  simple  or  complex,  and  this 
section  reviews  the  models  of  simple  and  complex  cells  that  are  employed  The  parameters  of 
<t>  are  selected  to  produce  orientation-tuning  and  length  summation  behaviour  resembling  either 
standard  or  special  complex  cells. 
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3.2.1  Simple  Cells 

We  model  the  response  of  simple  neurons  as  spatial  convolutions  against  the  image  and 
represent  the  RFs  by  Gabor  functions  (see  Appendix  1  Dobbins  et  al  1990)  This  class  of 
model  has  been  shown  to  provide  a  good  fit  to  the  spatial  RFs  of  simple  cells  (Jones  &  Palmer 
1987),  although  other  models  also  could  have  served  as  \<ell  (Parker  &  Hawken,  1988). 

3.2.2  Complex  Cells 

In  their  hierarchical  model  of  the  construction  of  visual  cortical  cell  types  Hubei  and  Wiesel 
(1965)  postulated  that  hypercomplex  cells  are  synthesized  from  complex  cells  which  in  turn  are 
composed  of  simple  components.  There  is  a  variety  of  evidence  incompatible  with  this  scheme 
in  its  simplest  form,  such  as  complex  cells  being  driven  monosynaptically  from  the  LGN  (Hoff¬ 
mann  &  Stone,  1971;  Singer  et  al.,  1975),  and  endstopped  (hypercomplex)  cells  appearing  in 
both  simple  and  complex  varieties  (Dreher,  1972;  Kato  et  al..  1978).  Although  cortical  neurons 
have  significantly  narrower  spatial  freqency  bandwidths  than  LGN  cells,  many  common  exper¬ 
imental  assays  would  fail  to  distinguish  complex  cells  with  staggered  simple  components  from 
cells  receiving  a  suitable  array  of  LGN  afferents.  Complex  cells  behave  spatially  as  if  they  receive 
input  from  simple  cells  with  slightly  shifted  positions.  Movshon  et  al..  (1978)  described  double 
bar  experiments  in  complex  cells  in  which  the  sign  and  magnitude  of  interaction  depended  on 
both  the  spatial  and  temporal  intervals.  They  argued  that  spatial  frequency  response  is  deter¬ 
mined  by  RF  subunits,  the  interaction  of  which  determine  the  directional  and  velocity  preference. 
Heggelund  (1981)  modeled  the  synthesis  of  complex  cells  from  overlapping  ON  and  OFF  center 
LGN  afferents  contributing  both  excitation  and  inhibition.  In  contrast,  Spitzer  and  Hochstein 
(1985)  treat  complex  cell  behaviour  as  arising  from  sums  of  half  wave  rectified  simple  cells.  Their 
model,  which  consisted  of  one  spatial  dimension  (across  the  RF)  and  time,  successfully  predicts 
the  temporal  response  of  many  simple  and  oomplex  cells  to  sinusoidal  grating  stimuli. 
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Standard  Complex 

Beginning  with  the  original  description  of  complex  neurons  (Hubei  &  Wiesel.  1962),  inves¬ 
tigators  have  described  their  diversity  in  orientation  tuning  and  length  summation  Our  present 
goal  leads  us  to  concentrate  on  the  spatial  aspect  of  response  dependency  in  classical  complex 
cells  (those  with  ON  and  OFF  responses  everywhere  in  the  RF)  and  not  to  attempt  to  portray 
the  features  accounting  for  speed  and  directional  preference  Therefore  we  build  complex-like 
cells  from  laterally  displaced  simple  cells,  rectifying  and  giving  an  individual  weighting  to  each 
The  complex  cell  response  is  given  by: 


n 

ffx  =£c’o(/?')  (4) 

i=l 

where  /?'  is  the  response  of  the  tth  simple  cell  and  c‘  its  weight.  For  simplicity  the  simple 
cell  components  are  equally  spaced  (shifted  less  than  1/4  the  spatial  wavelength)  and  Gaussian 
weighted  with  position.  In  a  purely  spatial  model  there  is  no  distinction  between  responses  to 
stimuli  being  turned  on  and  off.  Notice  that  for  a  particular  line  stimulus,  only  one  of  a  pair 
of  even  symmetric  simple  (or  LGN)  components  with  equal  and  opposite  RFs  would  respond. 
Therefore,  the  spatial  response  due  to  a  pair  of  half  wave  rectified  components  of  the  same 
position  but  with  opposite  contrast  sign  is  equivalent  to  the  full  wave  rectified  response  of  either 
of  the  pair.  For  computational  expedience,  in  the  simulations  we  use  only  ON  center  simple  cells 
with  full  wave  rectification. 


Figure  3  About  Here 

Figure  3  shows  line  weighting  functions  and  orientation  tuning  for  a  complex  model  instance 
and  one  of  its  simple  components.  The  complex  cell  responds  over  a  broader  area,  although 
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this  is  reduced  if  the  simple  components  are  half  wave  rather  full  wave  rectified  There  is  no 
corresponding  effect  on  orientation  tuning  Note  that  the  simple  and  complex  orientation  curves 
differ  insignificantly  in  tuning  breadth. 

The  orientation  width  of  the  complex  model  instance  is  approximately  the  same  as  its  simple 
components  (Figure  3c, d).  This  is  not  in  agreement  with  the  population  averages  reported  in 
many  investigations  (see  Orban  (1984),  for  a  review).  Among  the  possible  explanations  are:  (i; 
complex  cells  are  not  synthesized  from  simple  cells,  (ii)  the  simple  cells  that  compose  complex  cells 
have  broader  tuning  than  the  simple  population  as  a  whole,  (iii)  other  components  of  complex 
cells  broaden  the  orientation-tuning,  (iv)  subunit  interaction  broadens  orientation-tuning,  (v)  the 
simple  cell  components  have  randomly  perturbed  orientation  preferences,  broadening  the  complex 
cell  orientation  tuning  (suggestion  due  to  C  L.  Baker,  personal  communication),  (vi)  complex  cells 
exhibit  higher  firing  rates  than  simple  cells,  and  perhaps  operate  closer  to  saturation,  (vii)  the 
complex  population  is  less  homogeneous  than  the  simple  population  In  particular,  special  complex 
cells  have  considerably  broader  orientation  tuning  than  simple  or  standard  complex  cells  which 
differ  only  slightly  (Hammond  &  Pomfrett,  1989).  There  is  a  paucity  of  evidence  on  most  of 
these  points,  but  the  last  two  hypotheses  can  be  examined  with  the  activation  function  <J>(  )  in 
the  simulations.  By  setting  the  gain  and  saturation  appropriately  one  can  broaden  orientation 
tuning,  reduce  length  summation  and  examine  the  consequences  for  the  endstopped  model. 

Special  Complex 

We  briefly  describe  the  characteristics  of  special  complex  neurons  (Gilbert,  1977),  and  illus¬ 
trate  how  the  complex  cell  model  can  produce  them.  Special  complex  cells  are  characterized 
by  lack  of  length  summation  and  broad  orientation  tuning.  In  addition,  they  often  have  high 
spontaneous  activity  and  are  frequently  endstopped  (Gilbert,  1977;  Schwark  et'al..  1986;  Weyand 
et  al„  1986;  Hammond  and  Pomfrett  (1989)).  A  complex  model  cell  appears  to  be  of  the  special 
type  when  the  gain  is  chosen  so  that  the  activation  function  saturates  The  saturation  is  actually 
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subadditivity  or  occlusion, 


0(J1  -r  .r2)  <  <t>(.n)  -r  <t>(.!'2  ) 

and  can  be  biophysically  modeled  as  subadditive  synaptic  interaction  (Rail  et  al..  1967,  Rail. 
1970)  Schwark  et  al.,  (1986)  have  shown  that  the  supragranular  layers  are  a  necessary  input 
to  the  special  but  not  the  standard  complex  cells  of  Layer  5  in  the  cat  Therefore  subadditive 
interactions  could  occur  on  the  apical  dendrite  in  Layers  2-3,  or  via  the  supragranular  projection 
to  Layer  5. 


Figure  4  About  Here 

Figure  4  shows  length  and  orientation  tuning  curves  for  the  complex  cell  of  the  previous 
section  when  biased  to  exhibit  subadditivity.  The  length  tuning  responses  saturate  at  lengths 
considerably  less  than  the  RF  length  (61  pixels)  and  the  orientation  tuning  is  broadened. 

4.  Endstopped  Model  with  Simple  and  Complex  Components 


Figure  5  About  Here 


4.1  Simple  Center  and  Complex  Endzones 

We  begin  by  examining  the  response  of  an  endstopped  cell  in  which  the  small  cell  is  simple 
and  the  large  inhibitory  cell  is  complex.  Figure  5  shows  the  curvature-response  of  such  a  cell 
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as  a  function  of  the  width  of  the  curve  and  curve  orientation  In  Figure  5a  it  can  be  seen  that 
there  is  some  variation  in  response  with  the  width  of  the  curve  and  in  Figure  5b  it  can  be  seen 
that  there  is  some  response  recovery  for  low  curvatures  at  oblique  orientations  However  it  is  less 
than  half  the  response  to  high  curvatures  at  the  optimal  orientation  The  present  example  has 
excitatory  and  inhibitory  components  matched  in  center  frequency  but  achieves  robustness  with 
orientation  perturbation  with  increased  orientation  bandwidth  in  the  inhibitory  complex  cell.  In 
particular,  the  envelope  aspect  ratio  and  modulation  frequency  reduce  the  inhibitory  sidebands 
thus  broadening  the  orientation  bandwidth  and  DC  response  The  same  effect  could  have  been 
achieved  by  using  the  activation  function  to  bias  the  inhibitory  cell  to  look  special  complex-like 


Figure  6  About  Here 


4.2  Complex  Center  and  Complex  Endzones 

Figure  6  depicts  the  curvature  response  of  a  cell  in  which  both  small  and  large  components 
are  complex.  Perturbing  curve  orientation  (Figure  6b)  leads  to  somewhat  greater  oblique  low 
curvature  recovery  than  for  the  cell  in  the  previous  figure.  Figure  6d  repeats  the  experiment 
for  a  curved  edge.  There  is  no  response  to  curves  at  orientation  deviations  of  greater  than  10 
degrees  while  the  0  and  10  degree  curves  are  virtually  identical  In  Figure  6c  a  curved  line  and  a 
curved  edge  are  tested  at  all  positions  across  the  RF.  This  cell  gives  strong  curvature-dependent 
responses  to  both  curved  edges  and  lines,  although  not  in  a  position-independent  way.  In  both 
cases  the  maximum  response  is  displaced  from  the  RF  center,  but  the  edge  response  has  odd 
symmetry  being  maximal  when  the  edge  has  not  intruded  as  far  as  the  RF  center,  while  the  line 
response  is  roughly  even  about  the  RF  center.  If  one  measures  the  curvature-tuning  curve  at 
different  RF  positions  one  obtains  different  results  in  both  the  magnitude  and  shape  of  the  curve. 
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4  Endstopped  Model  with  Simole  and  Complex  Component* 
The  model  is  purely  spatial  while  real  neurons  have  temporal  dynamics  We  conjecture  that  the 
response  to  a  curve  swept  across  the  RF  in  the  preferred  direction  would  largely  integrate  across 
the  variation  in  the  curve  line  weighting  function 

Figure  7  illustrates  the  length  and  orientation  responses  of  the  same  endstopped  complex 
cell  Note  that  the  peak  of  the  length-tuning  curve  is  significantly  less  than  the  length  of  the 
short  excitatory  RF  (22  compared  to  35  pixels)  If  one  evaluates  the  orientation  response  with 
a  bar  of  this  length  one  obtains  a  curve  with  width  at  half  height  of  61  degrees.  In  contrast, 
evaluated  with  a  bar  of  length  35  pixels  the  width  at  half  height  is  50  degrees,  and  for  a  long 
bar  one  obtains  the  bifurcated  curve  shown.  Thus  the  broader  orientation  tuning  of  endstopped 
cells  than  their  nonendstopped  counterparts  (Kato  et  al.,  1978)  is  a  natural  consequence  of  the 
fact  that  for  endstopped  cells  the  measurements  are  made  at  shorter  lengths  relative  to  the 
underlying  discharge  region  (see  also  Dobbins  et  al.,  1989).  Similarly,  the  endzones  reduce  the 
apparent  length  summation  and  create  a  certain  amount  of  position-independent  response  along 
the  length  of  the  discharge  region  i.e.  .the  optima!  bar  can  be  moved  up  or  down  a  substantial 
amount  in  the  RF  and  still  elicit  a  strong  response.  This  is  also  a  common  property  among  the 
cells  of  V4,  many  of  which  have  both  antagonistic  surrounds  and  length-dependent  suppression 
within  the  discharge  region  (Desimone  &  Schein,  1987). 

We  have  seen  that  endstopping  can  broaden  measured  orientation  tuning  and  reduce  ap¬ 
parent  length  summation.  Since  special  complex  cells  are  frequently  endstopped  (Gilbert,  1977; 
Schwark  et  al,  1986;  Weyand  et  al.,  1986;  Hammond  and  Pomfrett  1989),  this  raises  the  ques¬ 
tion  of  whether  special  complex  cells  really  form  a  distinct  category.  It  may  be  that  endstopping 
is  partly  responsible  for  conferring  their  apparently  distinctive  position,  length  and  orientation 
properties  (see  section  on  Endstopping  Circuitry  also).  All  of  the  just-cited  studies  have  noted 
the  high  spontaneous  activity  of  special  complex  cells,  so  in  Figure  7c, d  the  activation  function  is 
introduced  (with  moderate  spontaneous  activity,  T  =  0.5)  to  the  output  of  the  small  component. 
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Figure  7c  shows  the  broadened  orientation-tuning  that  results  (width  at  half  height  100  degrees) 
Figure  7 d  shows  the  curvature  response  with  orientation  perturbation  The  oblique  low  curvature 
recovery  is  substantial.  Saturation  was  not  a  factor  in  these  results 


Figure  7  About  Here 

Employing  both  complex  components  and  the  activation  function  significantly 'increases  the 
dimensionality  of  the  parameter  space  and  the  range  of  behaviour  of  the  endstopped  model 
It  also  reduces  the  density  of  good  parameter  choices  for  stable  curvature  selective  response 
We  found  that  we  could  obtain  very  unusual  length,  orientation  and  line  weighting  functions 
when  combining  complex  cells  to  form  endstopped  cells.  For  example  in  one  instance  the  model 
cell  produced  a  strong  bifurcated  orientation  response  (usually  diagnostic  of  bandpass  curvature 
response)  but  gave  almost  no  response  to  curved  stimuli.  We  conclude  that  endstopped  complex 
cells  can  exhibit  stable  curvature  response  but  there  exists  no  necessity  that  they  do  so. 

These  explorations  imply  that  endstopped  complex  cells  can  produce  stable  curvature-selective 
response,  however  because  of  the  the  extraordinarily  broad  range  of  characteristics  that  complex 
cells  can  exhibit  there  are  many  more  ways  of  failing  to  achieve  stable  curvature  response  than 
to  achieve  it.  The  presence  of  endstopping  broadens  measured  orientation  tuning,  causes  some 
longitudinal  positional-independence  and  causes  the  optimal  length  to  be  significantly  less  than 
the  length  of  the  discharge  region. 

5.  Discussion 

5.1  Curve  Representation  in  Visual  Cortex 

Since  Hubei  and  Wiesel’s  first  explorations  in  visual  cortex  it  has  seemed  likely  that  cells  with 
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oriented  RFs  are  involved  in  the  description  of  visual  contours  In  this  section  we  recapitulate 
two  closely  related  perspectives  on  local  curve  representation  at  the  level  of  striate  cortex. 

We  begin  by  supposing  that  a  contour  is  represented  locally  by  a  truncated  Taylor  series 
approximation  The  difference  between  the  two  versions  depends  on  whether  emphasis  is  laid  on 
the  classical  RF  (discharge  region)  or  the  classical  RF  plus  inhibitory  zones.  In  the  first  version 
we  suppose  that  the  curve  (^(s))  is  represented  locally  as 

7(„<0  =  7(0)  +  7'(0)«  +  0)  (5). 

where  the  first  two  terms  are  position  and  orientation  estimates  at  arc  position  >  =  0,  and  the 
residual  /?2  depends  on  curvature.  In  this  scheme  we  suppose  that  the  representation  of  curves  is 
by  local  line  segments  and  that  the  purpose  of  the  residual  is  to  determine  the  largest  arc  length 
over  which  the  error  in  using  the  linear  (tangent- based)  approximation  is  less  than  some  value  B. 
Rearranging  and  taking  norms. 


-  7(0)  -  V(0)«|  <  *  (6). 

that  is,  we  wish  to  find  the  largest  value  of  s  such  that  the  inequality  is  satisfied  Put  differently, 
the  goal  is  to  determine  the  tangent  differential  (7'(0).s),  which  entails  finding  both  the  tangent 
and  the  maximal  arc  length.  In  this  view  discharge  regions  of  different  length  represent  assertions 
about  tangent  differentials  for  different  arc  lengths,  and  the  asymmetric  inhibitory  connections 
from  long  to  short  RFs  underlie  selection  of  the  longest  arcs  that  fit  the  data.  For  slowly  curving 
arcs  short  RFs  are  suppressed  by  long  RFs  which  represent  the  tangent  differential,  while  for 
rapidly  curving  arcs  long  RFs  respond  weakly  and  fail  to  suppress  the  short  RFs  which  represent 
the  tangent  differential.  According  to  this  viewpoint  curves  are  represented  at  this  stage  as 
overlapping  sets  of  line  segments  of  different  length. 

A  corollary  of  this  view  concerns  the  asymmetric  inhibition  from  long  to  short  RFs.  Clearly, 
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rf  a  tong  RF  cell  at  a  given  position  and  orientation  is  strongly  active,  the  activity  of  shorter  RF 
cells  at  that  position  and  orientation  is  redundant  Therefore  representing  a  curve  in  terms  of 
the  longest  segments  that  satisfy  some  error  bound  is  a  more  compact  representation  (see  Leclerc 
(1989)  for  a  development  of  minimal  descriptive  length  ideas  in  vision) 

In  the  second  version  the  inhibitory  endzones  are  considered  in  the  RF  description.  Endfree 
neurons  are  described  as  before  while  endstopped  neurons  represent  an  additional  assertion  about 
curvature.  Some  endstopped  cells  respond  selectively  to  the  sign  of  curvature  of  the  stimulus 
instead  of  simply  to  the  magnitude  of  curvature  (Dobbins  et  al.  1987  Versavel  et  al . ,  1990) 
supporting  the  stronger  position  on  curvature,  and  the  proposal  that  endstopped  cells  represent 
a  position-tangent-curvature  estimate  (Dobbins  et  al.,  1989)  Although  the  evidence  to  date 
suggests  that  the  majority  of  endstopped  neurons  are  not  selective  for  the  sign  of  curvature  this 
should  be  no  more  worrisome  than  the  fact  that  the  majority  of  cells  preferring  motion  are  not 
directional.  Nevertheless,  if  the  endzones  of  endstopped  cells  exhibit  phase-independent  inhibition 
as  a  variety  of  evidence  suggests  (Hubei  and  Wiesel,  1965;  Orban  et  al..  1979b;  Tanaka  et  a!., 
1987),  then  it  is  difficult  to  account  for  curvature  sign  selective  responses  in  a  simple  way.  Among 
the  possibilities  are:  (i)  not  all  endstopped  neurons  have  complex-like  endzones,  or  (ii)  curvature- 
sign  selectivity  arises  from  a  difference  in  temporal  properties  in  discharge  region  and  endzones. 
A  temporal  difference  would  imply  that  the  degree  of  curvature  sign  selectivity  would  be  speed 
or  temporal  frequency  dependent. 

The  endstopped  simple  model  is  dosely  related  to  the  fuzzy  derivative  notion  of  Koenderink 
and  van  Doom  (1987)  in  which  the  RFs  are  low  order  derivatives  of  Gaussians,  and  the  convolution 
results  represent  the  Taylor  series  coefficients  (k-jets).  The  principal  difficulty  with  this  stronger 
position  is  that  endstopped  cells  also  respond  well  to  short,  uncurved  arcs.  Indeed  endstopped  cells 
respond  well  to  both  short  uncurved  arcs  and  longer,  curved  ones.  Finally  although  endstopped 
neurons  resppnd  in  a  curvature-selective  way,  the  response  of  an  individual  neuron  varies  along 
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many  stimulus  dimensions,  and  the  response  of  an  endstopped  neuron  should  not  be  viewed  as 
delivering  a  number  representing  curvature  K  interested  in  explicitly  computing  curvature,  one 
can  combine  the  responses  of  RFs  using  the  formulas  of  calculus  (Koenderink  &  van  Doom. 
1987;  Koenderink  &  Richards,  1988).  However,  it  is  our  view  that  the  responses  of  endstopped 
and  endfree  cells  are  employed  in  curve  inference  networks  in  the  cortex  in  such  a  way  that  their 
orientation  and  curvature-selectivities  are  exploited,  but  without  explicitly  or  precisely  computing 
curvature  (Zucker  et  al . .  1989). 

Both  simple  and  complex  endstopped  neurons  exhibit  curvature-selective  responses  (Dobbins 
et  al.,  1987;  Versavel  et  al. .  1990),  and  the  differences  in  properties  between  the  two  suggest 
differences  in  functional  role.  Simple  cells  have  strong  spatial  phase  dependence  while  complex  cell 
response  is  dominated  by  elevation  of  mean  firing  rate.  If  building  global  descriptions  of  contours 
depends  on  interactions  between  neurons  with  nearby  RFs  (for  example  between  endstopped  cells 
with  RFs  tangent  to  a  common  circle),  then  the  position  or  phase-specificity  of  simple  RFs  would 
permit  more  spatio-temporally  selective  interactions  than  complex  cells 

5.2  Endstopping  Circuitry 

Hubei  and  Wiesel  (1965)  discussed  two  models  for  generating  hypercomplex  cells.  In  one 
the  endzones  are  synthesized  from  a  single  long  RF  neuron,  and  in  the  other  the  endzones  derive 
from  multiple,  displaced,  similarly-oriented  cells.  To  a  first  approximation  both  models  exhibit 
similar  behaviour  (orientation-dependent  end  inhibition)  with  finer  scale  differences  depending  on 
the  precise  structure  of  the  components  and  the  degree  of  nonlinearity  of  their  combination.  We 
have  focused  on  the  first  of  the  two  because  it  is  the  simplest  model  capable  of  explaining  the 
principal  data. 

We  mention  two  circuits  for  generating  endstopped  neurons.  In  one.  long  RF  neurons  in 
Layer  6  inhibit  shorter  RF  neurons  in  Layer  4  (see  McGuire  et  al.,  1984;  Bolz  &  Gilbert,  1986). 
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A  second  circuit  is  suggested  by  the  experiments  of  Weyand  et  al  (1986)  In  this  scheme 
superficial  layer  complex  cells  form  the  discharge  region  of  Layer  5  endstopped  special  complex 
cells  via  the  apical  dendrite  or  projection  to  Layer  5  The  endzones  could  also  result  from  this 
latter  projection  or  be  synthesized  on  the  laterally  spreading  basal  dendrites  in  Layer  5 

Hammond  and  Pomfrett  (1989)  find  approximately  half  of  their  sample  of  special  complex 
cells  (35  out  of  68)  to  be  nonendstopped.  On  the  other  hand,  in  their  investigation  of  Layer  5 
neurons  Weyand  et  al.,  (1986)  find  the  special  complex  property  to  be  highly  correlated  with  the 
presence  of  endstopping.  Their  striking  finding  was  that  the  supragranular  layers  are  a  necessary 
input  for  the  special  but  not  the  standard  complex  cells  of  Layer  5  They  consider  both  the 
possibility  that  Layers  2-3  are  necessary  to  drive  the  special  complex  cells,  and  that  these  layers 
serve  to  transform  a  standard  complex  cell  to  a  special  complex  cell  In  view  of  the  correlation 
with  endstopping  these  authors  suggest  that  both  the  discharge  region  and  endzones  are  supplied 
via  the  supragranular  layers.  As  suggested  above,  a  simpler  possibility  is  that  the  discharge 
region  is  derived  from  supragranular  inputs,  but  that  the  endzones  are  synthesized  on  the  basal 
dendrites  of  the  Layer  5  pyramids.  This  proposal  has  the  virtue  that  the  RF  center  is  derived  from 
a  process  (apical  dendrite)  or  processes  with  less  lateral  extent  than  whose  which  give  rise  to  the 
endzones.  It  was  observed  earlier  that  the  endstopped  complex  model  naturally  displays  some  of 
the  characteristics  of  special  complex  cells.  However,  also  note  that  convergence  of  multiple  Layer 
2-3  complex  cells  onto  an  electrically  short  piece  of  apical  dendrite  has  the  required  characteristics 
for  the  subadditive  synaptic  mechanism  discussed  earlier  which  could  underlie  the  special  complex 
property  independent  of  the  presence  of  endstopping. 

Finally,  the  simulations  show  that  a  complex  cell  whose  response  is  the  sum  of  input  simple 
cell  responses  does  not  exhibit  broader  orientation  tuning  than  its  simple  components  or  less 
length  summation.  However  both  of  these  will  occur  rf  the  inputs  are  combined  subadditively, 
and  this  seems  to  us  the  simplest  way  of  understanding  the  observed  differences  in  length  and 
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orientation  responses  between  standard  and  special  complex  cells  Hammond  and  Pomfrett  (1989) 
have  suggested  a  functional  distinction  to  complement  the  mechanistic  difference  suggested  by 
the  complex  model.  They  interpret  the  greater  directionality  and  broader  orientation  tuning 
of  special  complex  cells  in  terms  of  specialisation  for  motion  rather  than  contour  processing 
Although  it  is  reasonable  to  suppose  that  cells  specialised  for  motion  or  contour  analysis  are  biased 
to  different  operating  points,  unequipped  with  temporal  dynamics,  the  complex  model  used  here 
cannot  address  that  issue.  Also,  Versa vel  et  al.  (1990)  report  that  in  addition  to  neurons  that 
prefer  high  curvatures  (strongly  endstopped  cells)  and  those  preferring  low  curvatures  (length 
summating  nonendstopped  cells)  a  third  class  of  cells  respond  to  all  curvatures  Although  this 
class  was  diverse  in  orientation  tuning  and  degree  of  endstopping,  many  of  its  members  appear 
to  be  special  complex.  This  is  supported  by  the  laminar  distribution  found  We  note  that  when 
the  endstopped  complex  model  is  biased  to  exhibit  broader  orientation  tuning  and  less  length 
summation  (within  the  discharge  region)  it  naturally  exhibits  broader  curvature  response. 

5.3  Endstopping  and  Psychophysical  Curvature  Discrimination 

Watt  and  Andrews  (1982)  compared  the  curvature  discrimination  of  human  observers  to 
an  ideal  statistical  observer  and  found  that  the  relative  efficiency  of  the  human  increases  with 
orientation  range  up  to  about  40  degrees  and  then  diminishes.  Wilson  (1985)  also  examined 
curvature  discrimination  and  compared  the  results  to  the  performance  of  his  line  element  model 
—  a  point  processor  model  in  which  the  vector  of  responses  of  a  number  of  spatial  frequency  and 
orientation  selective  mechanisms  is  compared  for  different  patterns.  Wilson  and  Richards  (1989) 
conclude  that  the  model  corresponds  to  the  human  data  at  high  curvatures  but  not  at  low. 
For  bw  curvatures  they  advocate  pooling  responses  of  elements  along  the  tangent  direction,  a 
special  case  of  cocircular  tangent  interactions  (Parent  &  Zucker,  1989).  A  method  of  computing 
the  connection  compatibilities  (and  hence  interaction  strength)  between  discrete  tangent  and 
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curvature  estimates  in  nearby  local  coordinate  frames  has  been  developed  by  Iverson  (1988). 

Wilson  (1985)  tested  the  behaviour  of  the  line  element  model  with  curves  of  increasing  arc 
length  and  found  results  essentially  similar  to  Watt  and  Andrews  (1982)  However  one  curious 
and  unexplained  feature  of  the  latter's  results  did  not  appear  in  the  model  s  behaviour  In  the 
curvature  discrimination  data  a  peak  relative  efficiency  occurred  at  an  orientation  range  of  37.5 
degrees  (k  =  0.065  radians/arcminute)  close  to  the  predicted  value  of  40  degrees,  and  a  second, 
unexpected  peak  occurred  at  an  orientation  range  of  68  degrees  (».  =  0.12  rad/arcmin).  Because 
of  the  number  of  data  points  these  were  considered  to  be  approximately  40  and  80  degrees.  The 
secondary  peak  occurred  at  a  radius  of  curvature  of  8  arcmin  This  value  is  compatible  with 
the  smallest  oriented  foveal  RFs  found  in  primate  VI  (Schiller  et  al..  1976:  Parker  &  Hawken, 

1988)  when  one  takes  into  account  that  the  shortest  RFs  are  often  endstopped  and  hence  their 
discharge  region  length  would  be  expected  to  be  systematically  underestimated  (Dobbins  et  al.. 

1989) .  Our  hypothesis  is  that  the  unexpected  second  peak  is  due  to  the  smallest  endstopped 
units  —  those  that  subserve  the  highest  curvatures  to  which  the  human  visual  system  is  sensitive. 
This  hypothesis  leads  to  the  following  prediction.  If  one  repeats  the  experiment  in  which  for  an 
arc  of  fixed  length,  curvature  (and  hence  orientation  range)  are  varied,  at  different  arc  lengths, 
the  secondary  peak  mil  occur  at  the  same  curvature  value,  and  not  at  the  same  orientation  range 
value.  If  one  starts  with  too  long  an  arc  however  the  smallest  units  will  never  come  into  play  and 
the  discrete  nature  of  the  underlying  process  would  not  be  expected  to  emerge. 

Watt  and  Andrews  (1982)  also  observed  that  with  prolonged  fixation  arcs  of  high  curvature 
take  on  the  appearance  of  a  polygonal  curve.  This  is  attributable,  we  believe,  to  adaptation  of 
the  smallest  or  highest  curvature  units,  leaving  the  remaining  larger,  lower  curvature  units  to 
represent  the  curve.  An  indirect  test  involves  measuring  the  time  for  the  effect  to  set  in,  with 
and  without  pre-adaptation  to  very  short  lines.  We  predict  curvature-selective  adaptation  with 
uncurved  adapting  stimuli. 
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5  Discussior 


5.4  Conclusions 

In  generalising  the  endstopped  simple  model  to  include  complex  components  we  found  that 
the  special  complex  property  could  depend  on  both  the  presence  of  endstopping  and  occlusive 
or  subadditive  interaction  among  inputs  (abstracted  in  the  model  as  saturating  activation).  The 
complex  endstopped  model  exhibits  curvature-selective  responses  but  is  not  uniquely  responsive 
to  appropriately  oriented  and  curved  contours.  However  because  of  the  full  wave  rectification  of 
subunits  in  the  complex  model  the  endzone  inhibition  is  phase  independent  On  the  one  hand 
this  makes  the  inhibition  more  all-encompassing  than  if  the  endzones  were  synthesized  from  a 
simple  cell,  while  on  the  other  it  precludes  the  possibility  of  sign  of  curvature  selective  responses 
arising  from  a  spatial  phase  difference  between  the  discharge  region  and  endzones 
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7  Appendix  ] 


7.  Appendix  1. 

The  positive  part  function  <t >(/?;  p.  f")  is  closely  related  to  elementary  functions  When  T  =  1 
it  reduces  to  a  hyperbolic  tangent,  which  in  turn  can  be  written  as  a  difference  of  standard  and 
reversed  logistic  functions: 


<t>(R,pJ  =  1)  =  tanh{2R/p)  =  ^—^7  - 
Re-introducing  f  and  fixing  f  =  ,  the  postive  part  function  can  be  written  as. 


(.41) 


<t>(R:pJ)  = 


(.42) 


1  +  e-(R- Ro)/ f>  tRr-,R<-r 

where  the  first  term  is  a  sliding  logistic  function,  and  the  second  term  determines  the  amount  of 
compression  of  negative  values.  This  form  emphasizes  that  gain  (/>)  and  spontaneous  firing  (I") 
are  not  decoupled  (see  also  Figure  2). 
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8.  Figure  Legends 

Figure  1  Models  of  the  simple  and  complex  cell  components  Spatial  integration  is  followed 
by  a  sturating.  rectifying  nonlinearity. 

Figure  2.  The  generalized  rectification  function  o(R,p.  T)  R  is  the  input  and  as  I"  (written 
s'  for  spontaneous  in  figure)  varies  from  0  to  1  the  output  goes  from  half  wave  rectified  to 
unrectified,  p  controls  the  contrast  sensitivity  or  gain. 

Figure  3.  The  response  of  a  complex  cell  instance  and  one  of  its  simple  components  as  a 
function  of  line  position,  orientation  and  width.  The  simple  cell  parameters  are  described  by  an 
even  symmetric  Gabor  RF  with  length  61  pixels,  envelope  aspect  ratio  (AR)  2  0.  and  modulation 
frequency  u0  =  0.0 5  (5  pixels  =  1/4  wavelength).  The  complex  instance  has  3  simple  components 
with  spacing  3  pixels  and  weightings  given  by  a  Gaussian  with  a  =  4.  The  line  weighting  function 
(bar  response  as  a  function  of  position)  is  shown  for  long  bars  and  widths  of  3,  5,  and  9  pixels 
(solid,  dotted  and  dashed  curves,  respectively).  The  fourth  complex  cell  line  weighting  function 
(very  long  dashes)  illustrates  half  wave  rectification  of  subunit  responses  (line  width  9):  Notice 
the  apparent  reduction  in  RF  width  that  results.  The  orientation  response  is  for  bars  centered  on 
the  RF  and  rotated  in  steps  of  4  degrees  for  the  same  bar  widths 

Figure  4.  Length,  orientation  and  curvature  tuning  curves  for  a  complex  model  instance 
biased  for  subadditivity.  The  dotted  (p  =  65)  and  dashed  ( p  =  43)  curves  illustrate  two  degrees 
of  saturation  while  the  solid  curve  repeats  the  nonsaturating  characteristic  of  the  previous  figure. 
For  convenience  of  comparison  all  outputs  are  normalized. 

Figure  5.  The  response  of  an  endstopped  instance  in  which  the  small  cell  is  simple  ($ize:35, 
AR.2.0,  uo=0.05,  <^=2.8)  and  the  large  cell  is  complex  (size:61,  3  components  with  spacing=4, 
and  Gaussian  weighting  o  =  10.0.  Each  component  has  AR=3.0  and  t/0  =0.05 . )  a.  The  response 
to  (binary)  curves  varying  in  radius  from  0  to  200  in  steps  of  10  as  a  function  of  curve  width. 
The  curves  are  of  width  1,3, 5,7,  and  9  pixels  (solid,  dotted,  dashed,  long  dashed,  and  dot-dashed, 
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respectively)  b  The  response  to  curves  of  orientations  from  0  to  40  degrees  from  optimal  in 
steps  of  10  (curve  width  5).  The  solid  curve  corresponds  to  the  dashed  curve  in  part  a. 

Figure  6  The  response  of  an  endstopped  instance  in  which  both  small  and  large  component 
cells  are  complex.  Both  the  small  and  large  complex  cells  are  composed  of  3  simple  components 
of  size  35,  spacing  4.  The  small  and  large  cells  have  Gaussian  weighting  ^  =  4.0  and  10.0,  gains 

=  1.9  and  =  1.0,  and  aspect  ratios  of  1.0  and  3.0.  respectively  a  The  response  to 
(binary)  curves  varying  in  radius  from  0  to  200  in  steps  of  10  as  a  function  of  curve  width  (solid, 
dotted,  and  dashed  are  of  width  1,  5,  and  9  pixels  respectively)  b  The  response  to  curves  of 
orientations  from  0  to  40  degrees  from  optimal  in  steps  of  10  (curve  width  5)  The  solid  curve 
corresponds  to  the  dashed  curve  in  part  a.  c.  The  response  to  a  curved  line  (width  5  pixels,  solid 
curve)  and  curved  edge  (dotted  line)  of  radius  30  pixels  as  a  function  of  position  across  the  RF 
At  the  leftmost  edge  of  the  graph  the  curve  is  at  the  boundary  of  the  RF  and  curves  away  from 
the  RF,  at  the  center  it  is  centered  on  the  RF  and  at  the  rightmost  edge  is  on  the  boundary 
of  the  RF  curving  toward  the  RF.  d.  The  curvature  response  to  a  curved  edge  when  the  edge 
is  centered  on  the  RF.  The  curve  orientation  is  varied,  and  the  response  curves  for  orientation 
deviations  0  and  10  degrees  are  indistinguishable,  while  there  is  no  response  to  20  or  30  degree 
deviation. 

Figure  7.  a.  The  length  response  for  the  endstopped  complex  cell  of  the  previous  figure 
(line  width  5,  step  size  2).  b.  The  orientation  response  of  the  same  cell  for  lines  corresponding 
to  the  peak  of  the  length-tuning  curve  (22  pixels,  solid  curve),  the  length  of  the  excitatory 
center  (35  pixels,  dotted  curve),  and  longer  than  the  discharge  region  plus  endzones  (80  pixels, 
dashed  curve),  c.  The  orientation  response  when  the  positive  part  function  is  employed  with 
moderate  spontaneous  firing  (I"  =  0.5,  p  =  40)  for  the  excitatory  component  (solid,  dotted,  and 
dashed  curves  show  the  response  of  the  endstopped  cell  and  its  small  and  large  components, 
respectively),  d.  The  curvature  tuning  curve  for  the  cell  (with  spontaneous  firing  as  in  part  c.)  as 
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curve  orientation  is  perturbed  (0,10,20,30  degrees  for  the  solid,  dotted  dashed  and  long  dashed 
curves,  respectively). 
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A  MEAN  FIELD  MODEL  OF  OPTIC  FLOW  ESTIMATION 

Allan  Oobbinst  Steven  W.  Zuckert  Max  S.  CvnaderS 
fMcGill  University,  Montreal  §University  of  British  Columbia,  Vancouver. 

We  address  two  questions.  How  can  the  visual  system  obtain  reliable  estimates 
of  the  optic  flow  held?  How  can  local  motion  and  disparity  estimates  be  related 
to  the  shape  of  visible  surfaces?  From  the  observed  properties  of  visual  cortical 
cells  we  build  model  neurons  and  local  circuits  that  are  evaluated  in  dynamic 
simulations.  Neurons  in  the  middle  temporal  visual  area  (MT)  are  direction  and 
speed  selective  and  many  also  have  direction  and  speed-dependent  antagonism 
between  the  receptive  field  (RF)  center  and  surround.  Given  an  ensemble  of  such 
cells  with  RFs  spanning  a  range  of  sizes,  it  is  possible  in  principle  to  obtain  maximal 
mean  field  approximations  of  the  optic  flow  field  —  the  largest  neighbourhoods 
around  each  point  in  which  the  field  is  well-approximated  as  uniform  translation. 
The  advantage  of  this  approach  is  that  estimation  of  the  rate  of  spatial  variation 
of  the  field  yielos  improved  signal-to-noise  ratio  without  blurring  across  distinct 
spatial  regions.  Although  center-surround  speed  bandpass  neurons  should  respond 
well  to  spatially  restricted  translation,  or  to  appropriately  curving,  accelerating,  or 
discontinuous  flows;  they  provide  no  information  about  which  of  these  situations 
is,  in  fact,  the  case.  Estimation  of  the  large  scale  first  order  variation  of  the  flow 
field  appears  to  occur  subsequently  (Saito  et  a/,  1986),  however  it  is  the  second 
order  variation  that  is  related  to  surface  shape.  We  shall  show  how  neurons  may 
be  organized  to  provide  coarse  information  about  the  shape  of  surfaces  from  the 
second  order  variation  of  the  optic  flow  field  and  from  binocular  disparity. 
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Abstract 

In  this  paper  we  consider  the  problem  of  obtaining  reliable 
estimates  of  the  optic  Row  field  and  its  variation.  We  shall  ar¬ 
gue  that  circuitry  in  the  middle  temporal  visual  area  (MT)  is 
arranged  to  provide  mean  field  approximations  of  the  optical 
flow  field  —  the  maximal  neighbourhood  over  which  the  local 
field  can  be  well-approximated  as  uniform,  parallel  translation. 
In  addition,  the  way  the  mean  field  approximation  varies  with 
neighbourhood  size  and  position  is  informative  about  the  vari¬ 
ation  of  the  flow  field.  Further,  we  show  how  receptive  fields 
can  be  synthesized  to  represent  the  first  order  differential  in¬ 
formation  about  the  flow  field.  The  same  ideas  are  applicable 
to  the  binocular  disparity  vector  field,  and  lead  us  to  predict 
the  existence  of  extrastriate  neurons  that  are  specialised  for 
the  local  shear  and  stretch  of  the  disparity  and  motion  vector 
fields. 


1.  Introduction 

Our  goal  is  to  understand  how  the  brain  estimates  surface 
shape  from  the  visual  information  available  to  it.  This  problem 
hid  played  a  central  role  in  theories  of  vision  (Gibson.  1950; 
Marr.  1982).  Here  we  shall  consider  those  cases  in  which  there 
are  vector  fields  associated  with  the  surfaces:  the  motion  flow 
field  and  the  binocularity  disparity  field. 

Koenderink  has  repeatedly  emphasized  the  value  of  the 
differential  structure  of  vector  fields.  In  the  optic  flow  field 
these  are  invariant  with  a  constant  translation  whether  due 
to  object  movement  or  an  eye  movement.  Thus  the  appropri¬ 
ate  setting  for  the  first  order  differential  information  is  affine 
space.  In  both  the  binocular  disparity  field  and  the  motion 
flow  field,  all  the  information  about  surface  orientation  is 
contained  in  the  deformation  component  (Koenderink  !i  van 
Doom.  1975.1976;  Koenderink.  1986]. 

In  this  paper  we  shall  show  how  the  brain  could  estimate 
components  of  the  gradient  of  the  optic  flow  field.  We  shall 
argue  that  instead  of  simply  differentiating  the  flow  field  and 
combining  the  partial  derivatives,  the  circuitry  of  the  middle 
temporal  cortex  (MT)  determines  maximal  mean  field  esti¬ 
mates  —  the  largest  convex  regions  where  the  flow  is  well- 
approximated  as  parallel.  If  such  patches  can  be  obtained, 
certain  simplifications  follow  in  the  gradient  computations. 


In  the  next  section  we  treat  the  generic  features  of  visual 
cortical  architecture  that  we  think  underlie  the  computations 
that  are  described  subsequently. 

2.  Visual  Cortex:  Estimators  and  Architecture 

In  this  section  we  briefly  review  some  features  of  the  or¬ 
ganization  of  the  primary  visual  cortex  (VI)  and  the  middle 
temporal  area  (MT)  that  are  germane  to  the  developments 
that  follow. 

Primary  visual  cortex  is  organized  into  columns  of  cells 
which  share  their  preferred  orientation  (Hubei  It  Wiesel.  1962], 
Thus  each  retinal  point  is  multiply  represented  in  a  set  of 
columns  spanning  orientation  space.  In  MT.  an  area  which 
a  variety  of  evidence  indicates  is  specialised  for  motion  anal¬ 
ysis.  the  cortex  is  also  organised  into  columns  (Albright  et 
al.  1984).  In  this  case  the  units  in  a  column  share  a  pref¬ 
erence  for  direction  of  motion.  In  addition.  MT  neurons  are 
speed-selective  with  individual  cells  exhibiting  a  broad  range 
of  preferred  speeds  and  bandwidths  (Maunsell  it  Van  Essen. 
1983). 

Some  of  the  cells  in  VI  exhibit  an  orientation-specific,  an¬ 
tagonistic  relationship  between  the  receptive  field  center  and 
zones  beyond  the  ends  of  the  excitatory  receptive  field.  Orig¬ 
inally  termed  hypercomplex  when  they  were  first  described  in 
extrastriate  cortex  [Hubei  and  Wiesel.  1965).  these  neurons 
exhibit  a  differential  response  with  stimulus  length,  and  are 
commonly  referred  to  as  endstopped.  We  have  previously  de¬ 
scribed  a  computational  model  of  endstopped  cells  which  is 
length  selective,  but  which  is  curvature-selective  in  addition. 
Hubei  and  Wiesel  recognized  that  endstopped  cells  should  re¬ 
spond  well  to  highly  curved  stimuli  that  avoid  the  end  zones, 
and  recent  neurophysiological  experiments  demonstrate  that 
endstopped  cells  are  curvature-selective  [Dobbins  et  al.  1987). 
In  area  MT  many  cells  exhibit  a  direction  and  speed  spe¬ 
cific  antagonism  between  center  and  surround  [Allman  et  al. 
1985).  They  also  exhibit  differential  response  with  flow  field 
variation.  In  the  following  section  we  develop  a  view  of  the 
estimation  carried  out  by  MT  circuitry. 

3.  The  Mean  Field  Model 

Although  field  theories  can  be  formulated  in  differential 
(point)  or  integral  form,  it  makes  little  sense  for  compute- 


tional  vision  theories  to  be  formulated  in  differential  form. 
There  are  several  reasons.  One  is  that  because  of  the  uncer¬ 
tainty  relation  between  a  signal  and  its  derivative,  one  cannot 
obtain  information  about  the  infinitisimal.  differential  infor¬ 
mation  in  a  flow  field.  A  second  constraint  is  imposed  by 
the  representational  precision  of  the  local  estimators:  shrink¬ 
ing  the  temporal  neighbourhood  shrinks  the  data  precision 
correspondingly.  Thirdly,  the  surface  features  in  motion  can 
change  in  size,  orientation,  and  shape.  For  example,  if  we 
modeled  the  time-variation  of  an  image  feature  as  a  wave 
group,  its  motion  is  characterised  by  both  phase  (ui/k)  and 
group  (du/dfc)  velocities. 

Since  the  visual  system  cannot  have  access  to  the  point 
information,  it  must  employ  spatio-temporal  integral  mea¬ 
sures.  An  essential  part  of  its  task  is  determining  the  neigh¬ 
bourhood  size  over  which  a  particular  order  estimate  holds. 
Here  we  propose  an  approach  to  how  the  part  of  the  cor¬ 
tex  specialised  for  motion  analysis  determines  the  best  linear 
approximation  to  the  flow  over  various  neighborhood  sizes. 


Figure  1  Four  flow  patterns,  ont  of  whkti  Is  dlKOfl- 
tlnuous.  all  of  which  are  w«ft-ipproxlmit«d  by  uniform 
translation  ovur  tha  small  neighbourhood  disk  but  not 
over  the  large  one. 


3.1  Definition  of  Terme 

By  the  optic  flow  field  we  shall  mean  the  mapping,  v  : 
RJ  -»  R1.  from  retinal  coordinates  to  TM  the  tangent  bundle 
of  the  flow  field.  Hence  the  image  of  x  at  time  f0  is  the  velocity 
vector  vfajlisto  TVs  direction  field  is  a  mapping  from  retinal 
coordinates  to  the  direction  of  the  flow  velocity  (x  *-*  v/||v||). 
By  a  mean  field  approximation  of  the  flow  field  we  shall  mean 
the  vector  or  differential  1-form  that  is  the  estimated  average 
translation  velocity  over  a  specified  disk  (D(r,x))  around  a 
point.  Naturally,  this  comprises  both  the  average  speed  and 
direction. 

3.2  Initial  Measuremente 

Our  goal  is  to  estimate  the  affine,  linear  structure  of  the 
flow  field.  Toward  this  end  we  assume  the  availability  of  two 
types  of  local,  speed  selective,  direction  selective  units. 


broedband  unite  These  are  calls  which  have  a  speed  band¬ 
width  of  two  to  three  decades  (r^). 

narrowband  unite  These  are  celts  which  have  a  spend 
bandwidth  of  two  to  four  octaves  )- 

The  narrowband  units  span  the  range  of  speeds  to  which  the 
system  is  sensitive,  and  both  types  of  units  cover  the  set 
of  retinal  directions  and  the  retina  itself.  We  shall  not  be 
concerned  here  with  detailed  description  of  these  cells,  but 
mention  that  they  are  meant  to  correspond  to  Area  VI  and 
V2  units. 


3.3  Pooling  Responses 


At  the  next  stage  the  responses  of  many  of  these  units  are 
pooled,  and  it  is  to  the  function  of  this  process  that  we  now 
turn.  Let  us  sum  over  a  convex  retinotopically-mapped  region, 
the  responses  of  a  set  of  narrowband  units  whose  receptive 
fields  cover  the  region.  All  the  units  are  tuned  to  the  same 
direction  (#,)  and  speed  of  movement  (vy).  Although  the 
image  and  inputs  are  discrete,  for  convenience  we  shall  refer 
to  integration.  The  spatial  integral  of  the  unit  responses  over 
a  disk  D(r,  x)  is: 

*£iy  (»•  *)  =  JD  V/  (Vl  *)dA  I1) 

This  measures  tha  degree  of  correspondence  of  the  flow  field 
to  a  uniform,  parallel  flow  in  the  direction  to  which  the  units 
are  tuned.  Normalizing  by  area,  we  obtain  a  space-averaged 
measure  of  the  projection  over  the  retinal  area  in  question: 


<W>  -  iflgj’M- 


P) 


Alternatively,  summing  over  the  broadband  units  fry.), 
one  obtains  a  measure  that  varies  with  direction  but  not  with 
speed.  Therefore  a  pattern  translating  in  the  right  direction 
but  with  inhomogeneous  speed  will  elicit  strong  response. 

Wt  noted  above  that  the  distribution  of  response  with  di¬ 
rection  contains  important  information  about  the  field  struc¬ 
ture.  but  it  also  should  be  clear  that  the  set  of  directional 
spatial  averages  over  a  particular  neighbourhood  does  not 
uniquely  determine  the  flow  field  (see  Figure  1).  Therefore 
we  investigate  the  significance  of  the  variation  in  the  mean 
field  estimates  with  integration  area. 


3.4  Analyels  of  Flow  Field  Variation  with  Area 

Now  we  wish  to  examine  how  the  integrated  response 
varies  with  srea.  For  convenience  we  shall  simplify  the  nota¬ 
tion.  ignoring  the  dependence  of  A  on  position  and  the  flow 
field,  and  simply  refer  to  its  area-dependence  J?(A).  Either  R 
or  7?  can  be  employed  for  the  analysis.  It  is  assumed  that  the 
local  unit  measures  are  nonnegative,  implying  that  SR  >  0. 
Case  1.  R(A)  constant,  a.  K  A(A)  =  0.  than  there  is  no 
component  of  the  flow  field  in  the  direction  #.  b.  If  A(A)  >  0. 
then  K  -*  0  ss  1/A.  For  the  minimal  disk  of  area  A0  there 


is  *  response  but  there  is  no  contribution  from  the  surround¬ 
ing  ares.  Thus  there  is  either  no  field  outside  A«  or  it  is 
orthogonal  to  9. 

Case  2.  J?(d)  increasing.  There  is  a  contribution  to  the 
integral  outside  A0.  There  are  three  principal  cases  according 
to  the  sign  of  the  second  derivative  of  R  with  respect  to  A. 

a.  d?R/dAi  >  0.  R  inflects  up  implying  that  the  field  corre¬ 
sponds  better  to  9  over  a  larger  area.  This  occurs  when  the 
average  field  moves  toward  the  estimators  either  in  direction 
or  speed.  Generally.  9  would  not  be  the  mean  field  estimate  at 
smaller  areas.  Thus  this  case  generally  represents  a  change 
in  the  mean  field  approximation  with  area. 

b.  d1R/dAi  =  0.  R  increases  proportional  to  area.  This 
indicates  that  the  field  is  not  changing  systematically  with 
respect  to  9.  It  does  not  necessarily  imply  that  9  is  the  best 
mean  field  estimator  however.  For  example,  if  the  field  is  a 
uniform  field  translating  at  an  acute  angle  with  respect  to 
direction  I.  R  increases  linearly  with  A.  the  slope  depending 
on  the  relative  angle.  In  this  case  there  should  be  another 
direction  that  is  the  best  mean  field  estimate  at  all  areas  for 
which  this  condition  holds. 

c.  d*R/dA J  <  0.  R  inflects  downward,  implying  that  the 
estimators  conform  better  to  the  field  over  smaller  areas.  If 
9  is  the  mean  field  direction  for  small  A  but  not  large  A  then 
the  field  direction  is  changing  (either  smoothly  or  abruptly). 
If  9  is  the  mean  field  direction  for  both  small  and  large  A.  then 
the  field  is  either  less  parallel  (the  distribution  of  R  over  the 
set  of  directions  is  broadening  with  increasing  A),  or  speed  is 
changing. 

Note  that  we  did  not  specify  whether  integration  was  tak¬ 
ing  place  over  narrowband  units,  broadband  units  or  both. 
Suppose  that  separate  integrations  are  done  for  the  narrow¬ 
band  units  tuned  to  separate  speeds,  as  well  as  for  the  broad¬ 
band  units.  Since  the  latter  are  concerned  only  with  direction 
and  not  with  speed  (over  a  broad  range)  their  change  in  re¬ 
sponse  can  only  be  due  to  a  change  in  direction.  In  contrast 
the  narrowband  unit  integration  changes  with  both  speed  and 
direction.  However  the  difference  between  the  change  in  re¬ 
sponse  due  to  direction  and  speed,  and  that  due  to  direction, 
is  the  change  in  response  due  to  speed  change.  Assuming 
quasi-orthogonality  (e.g.  a  stimulus  halfway  between  centers 
of  two  adjacent  direction  or  speed  classes  elicits  a  half  re¬ 
sponse  from  each),  summing  over  speed  and  direction  classes, 
and  subtracting  broad  from  narrowband  totals,  one  could  in 
principle  partition  the  variation  into  speed  and  directional  vari¬ 
ation.  W«  note  in  passing  that  the  convexity  detector  model 
of  Nakayama  and  Loomis  [1974]  involved  summing  (center- 
surround  velocity  filters)  over  direction  to  detect  speed  dis¬ 
continuities  in  the  optic  flow  field.  The  interpretation  of  the 
responses  of  such  detectors  is  problematic  in  a  world  with 
nonplanar  surfaces  however. 

3.5  MT  Receptive  Field  Organization 

Now  we  shall  attempt  to  connect  the  process  described 
above  to  what  is  known  of  cortical  organization.  We  are  in¬ 
terested  in  the  value  of  R{A)  and  its  first  two  derivatives. 


However  it  is  easily  shown  that  this  information  is  contained 
in  the  first  variation  of  7?.  To  see  this,  consider  the  difference 
in  the  value  of  Jl  over  disks  of  area  a  and  b  (b  >  e).  Some 
rearrangement  yields: 

E(fc)_*(a)  =  =  »-«[*(») -*(e)  R(A) 

b  a  Alb— a  a 

(3) 

The  terms  in  braces  are  the  average  projections  of  the  flow 
field  onto  the  annulus  of  outer  radius  b  and  the  disk  of  radius 
a.  Therefore  the  whole  expression  is  the  difference  of  the 
field's  projection  onto  the  annulus  and  the  disk  it  surrounds. 
Hence  instead  of  evaluating  the  infinitisimal  second  derivative 
of  R.  the  finite  first  variation  of  the  space-average  measure  is 
used. 

Consider  a  finite  difference  scheme  in  which  7F(A)  is  es¬ 
timated  at  intervals  e.g..  A0.  2A0.  4 A*,...  Then  the  differ¬ 
ence  of  the  successive  area-normalized  integrals  is  computed: 
7?(Aa)  -  5(2 A*)  and  so  forth.  In  physical  terms,  we  suppose 
that  there  is  a  set  of  neurons  the  responses  of  which  corre¬ 
spond  to  7?(A«).  S(2A0)  and  to  forth,  and  further,  that  are 
neurons  which  represent  the  differences  of  successive  terms  in 
this  sequence.  These  neurons  encode  the  variation  of  the  flow 
field  over  successive  area  increments.  In  addition  to  capturing 
the  rate  of  change  of  the  flow  field,  this  circuitry  can  also  be 
exploited  to  determine  the  largest  neighbourhood  over  which 
the  field  is  well-approximated  by  parallel  translation.  In  par¬ 
ticular  if  9  is  the  direction  of  maximum  response  over  a  disk 
A,-  and  Xy  (A,-)  -  5y(A,+ j)  <  0.  then  A,  can  be  replaced  in 
subsequent  computations  by  A,+1. 

It  is  natural  to  suppose  that  the  difference  computation 
occurs  within  a  cortical  column.  For  most  smoothly  varying 
patterns,  all  receptive  field  centers  less  than  some  size  are 
strongly  inhibited  by  their  surrounds.  Then  at  some  point  in 
the  sequence  there  is  a  center  which  is  both  strongly  activated 
and  not  strongly  inhibited.  The  area  of  the  center  of  this  cell 
is  the  maximal  patch  area. 

Note  that  although  the  magnitude  of  the  variation  of  the 
flow  field  surrounding  a  point  can  be  estimated  in  this  way. 
this  scheme  does  not  represent  the  nature  of  the  variation. 
There  would  seem  to  be  two  possibilities.  The  first  is  that 
the  form  of  the  variation  is  distributed  over  the  set  of  direction 
and  speed  integrals  over  patches  of  the  field.  Alternatively, 
particular  forms  of  variation  may  be  explicitly  represented  in 
the  response  of  individual  neurons.  In  the  next  section  we 
shall  consider  how  this  might  be  accomplished. 

4.  Estimating  Differential  Flow  Structure 

In  this  section  we  consider  how  neurons  could  estimate 
the  spatial  gradient  of  the  optic  flow  field.  The  circuitry  de¬ 
scribed  in  the  last  section  is  used  to  determine  the  largest 
neighbourhood  in  which  the  flow  can  be  treated  as  parallel. 
The  mean  field  circuitry  imposes  a  choice  of  coordinate  sys¬ 
tem  (p,n).  representing  the  parallel  and  normal  components 
of  the  flow,  v  =  (vp.vn).  and  selects  a  neighbourhood  size 


Figure  2  A  tchtmilk  rcpresentitlon  of  two  direction- 
selective  receptive  fields  with  antagonistic  center  end 
surround.  The  surround  Is  split  into  two  subiones  in 
each  case. 

over  which  the  normal  component  is  negligible.  Hence  over 
region  A.  v  «  (vp,  0). 

Consider  the  first  order  variation  of  the  flow  with  respect 
to  vp.  The  variation  is  decomposed  into  ^  and  How¬ 
ever.  keep  in  mind  that  the  estimates  are  always  integrals  over 
area.  Wfe  now  propose  that  these  quantities  can  be  estimated 
with  center-surround  mechanisms  differing  in  only  one  respect 
from  those  described  in  the  last  section.  Suppose  that  the 
surround  is  divided  into  halves.  In  one  half  the  constituent 
units  are  low  pass  for  speed  and  in  the  other  hald  high  pass. 
Figure  2  shows  examples  in  which  the  division  is  parallel  and 
perpendicular  to  the  directional  preferences  of  the  center  and 
surround.  The  additional  requirement  is  that  the  rolloff  rates 
of  the  low  and  high  pass  mechanisms  are  equal  and  opposite. 
Before  proceeding,  it  should  be  mentioned  that  the  surround 
units  can  actually  be  bandpass,  the  important  thing  is  that 
they  are  lowpass  and  highpass  in  the  window  of  speeds  to 
which  the  center  is  sensitive.  Figure  3a  shows  the  arrange¬ 
ment.  All  the  units  (both  lowpass  and  highpass)  comprising 
the  surround  are  summed  as  before,  and  the  area-normalized 
value  is  subtracted  from  the  value  for  the  center. 

Now  consider  the  response  of  this  mechanism  to  flow  pat¬ 
terns.  For  a  uniform,  translating  pattern,  the  cell  behaves  as 
before  -  the  surround  cancels  the  response  of  the  center.  This 
is  because  the  equal  slope  condition  ensures  that  the  sum  of 
the  low  and  highpass  units  is  invariant  over  a  range  of  speeds. 
However,  its  response  is  quite  different  to  patterns  with  spa¬ 
tial  variation  of  speed.  Suppose  that,  in  the  coordinate  frame 
(p,n).  the  velocity  field  is  given  by:  v  =  (vo+mp,0)  where  m 
is  the  spatial  gradient  of  the  speed.  The  cell  will  respond  pro¬ 
portional  to  m.  To  see  why.  note  that  we  could  equally  have 
described  the  surround  organization  as  a  difference  of  two 
spatially  segregated  low  pass  (or  high  pass)  mechanisms  of 
identical  rolloff.  Thus  the  surround  represenU  a  first  spatial 
difference  of  speed,  and  responds  proportional  to  the  speed 
gradient.  This  is  shown  in  Figure  3b.  The  center  serves  to 
establish  that  the  field  is  sufficiently  parallel  and  has  average 
speed  within  some  range,  and  the  surround  modulates  it  pro¬ 
portional  to  the  gradient.  Observe  that  swapping  the  low  and 
high  pass  regions  reverses  the  sign  of  gradient  to  which  the 
cell  responds. 

A  cell  with  the  surround  split  as  above  a  interested  in 
the  speed  change  in  the  direction  of  the  flow.  If  the  surround 


Figure  3  a.  TM  specs  dependence  of  the  response  of 
the  center  (C).  and  the  two  halves  of  the  surround  (L 
and  H)  of  a  hypothetical  center-surround  cefl.  b.  The 
response  of  the  cell  with  the  gradient  of  speed.  In  the 
direction  normal  to  the  axis  of  division  of  the  surround. 

These  art  not  quantitative  simulations,  manly  a  quali¬ 
tative  sketch. 

is  divided  parallel  to  the  preferred  direction,  the  cell  responds 
proportional  to  the  (parallel  component  of)  speed  change  nor¬ 
mal  to  the  flow  direction,  that  is.  proportional  to  the  shear 
of  the  field.  Generally,  the  shear  depends  on  as  well. 
To  the  extent  that  the  mean  field  circuitry  has  a  chosen  a 
neighbourhood  size  such  that  vn  =  0  the  contribution  of  this 
term  is  negligible.  Simulations  are  in  order  to  determine  the 
performance  limitations  of  this  scheme. 

S.  Summery  end  Conclusions 

1.  Integration  of  the  broadband  units  underlies  the  com¬ 
putation  of  the  mean  direction  field,  and  the  differential  of 
this  field  describes  the  rate  of  directional  change  with  area. 

2.  In  contrast,  integration  of  the  narrowband  directional 
units  underlies  estimation  of  the  mean  velocity  field,  and  its 
differential  describes  the  rate  of  change  of  velocity  (both  di¬ 
rection  and  speed)  with  area. 

3.  The  result  of  the  mean  field  estimation  process  is  a 
set  of  patches  of  varying  size  covering  the  Row  field.  Each 
patch  has  an  associated  velocity  vector. 

4.  To  the  degree  that  the  mean  field  circuitry  succeeds 
in  choosing  a  neighbourhood  size  such  that  the  normal  com¬ 
ponent  of  velocity  is  negligible  over  the  neighbourhood,  the 
vector  field  is  locally  reduced  to  a  scalar  field,  and  the  gradient 
of  the  field  is  reduced  from  a  second  order  tensor  to  a  first 


order  tensor  (a  vector).  The  consequence  is  a  reduction  in 
complexity  in  the  sense  of  the  number  of  numbers  associated 
with  an  estimate. 

5.  Although  space  does  not  permit  its  development  here, 
it  b  possible  to  develop  estimates  of  the  components  of  the 
disparity  gradient  using  opposite  slope  estimators  (near  and 
far  cells},  in  an  analogous  manner  to  that  described  here  for 
the  components  of  the  optic  flow  gradient. 

6.  The  ideas  described  here  have  so  far  neither  had  their 
quantitative  plausibility  established  in  simualtions.  nor  their 
truth  refuted  in  physiological  experiments.  Both  courses  of 
action  will  be  pursued. 
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On  the  Evolution  of  Curves  via  a  Function  of  Curvature,  I: 

The  Classical  Case 
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Abstract 

The  problem  of  curve  evolution  as  a  function  of  its  beat  geometry  arises  naturally  in  many 
physical  applications.  A  special  case  of  this  problem  is  the  curve  shortening  problem  which  has 
been  extensively  studied.  Here,  we  consider  the  general  problem  and  prove  an  existence  theorem 
for  the  classical  solution.  The  main  theorem  rests  on  lemmas  that  bound  the  evolution  of  length, 
curvature  and  how  far  the  curve  can  travel. 

Keywords:  Curve  Evolution,  Deformation,  Curve  Shortening,  Flame  Propagation,  Curvature 
Bounds. 
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1.  Introduction 

In  this  paper,  we  study  certain  evolution  equations  of  embedded  plane  curves  where  the  speed  of 
the  deformation  is  a  function  of  the  curvature,  and  whose  direction  is  in  the  normal  direction. 

The  curve  evolution  problem  is  relevant  in  applied  sciences.  The  study  of  immersed  closed 
curves  evolving  as  functions  of  their  curvature  has  been  carried  out  lor  crystal  growth  [Langer, 
1980;  Ben- Jacob  et  a/.,  1983],  flame  propagation  [Sethian,  1985;  Sivashinsky,  1977;  Osher  and 
Sethian,  1988],  and  curve  shortening  [Gage  and  Hamilton,  1986;  Grayson,  1987].  We  would  like 
to  investigate  properties  of  the  classical  solutions  of  these  evolution  equations.  As  pointed  out  by 
the  referee,  similar  work  has  been  done  independently  by  [Angenent,  1989].  Finally  the  equations 
we  use  for  the  curvature  of  a  plane  curve  evolving  according  to  the  general  law  given  below  are 
classical;  see  e.g.  [Brower  et  al.,  1984].  In  a  sequel  we  plan  to  consider  the  weak  solutions  when 
shocks  develop  [Lax,  1973;  Lax,  1971;  Osher  and  Sethian,  1988]. 

The  research  has  been  motivated  by  the  study  of  certain  problems  in  computer  vision  [Kimia 
et  al.,  1990;  Kimia,  1989].  Indeed,  we  are  interested  in  studying  the  problem  of  shape  perception, 
and  explicitly  in  the  reconciliation  between  the  parts  versus  protrusions  dichotomy  in  computer 
vision.  This  paper  is  primarily  concerned  with  providing  a  rigorous  basis  to  these  results. 

The  authors  would  like  to  thank  Professor  Gprian  Foias  for  some  very  helpful  conversations 
on  this  paper,  especially  about  Lemma  4.2.1. 

1.1  Notation 

We  will  now  set  up  some  of  the  basic  notation  and  concepts  which  we  will  need  in  what  follows. 

Let  C(s,t)  :  Sl  — ►  R2  be  a  family  of  embedded  curves  where  t  denotes  time  and  s  parame¬ 
terizes  each  curve.  We  assume  that  this  family  evolves  according  to  the  evolution  equation 

^  =  a (1.1) 

C{»,0)=  &(>),  (1.2) 

where  N  is  the  outward  normal,  k  is  the  Gaussian  curvature,  and  a,  /?  are  arbitrary  functions. 
For  each  deformation  {a,/?},  there  exists  another  deformation  {0,/?'}  such  that  the  resulting 
traces  of  curves  are  equivalent  [Gage,  1986].  Furthermore,  we  constrain  the  deformations  to  be 
determined  by  the  local  geometry  of  the  curve,  i.e.,  /?  should  be  a  function  of  curvature  [Kimia, 
1989].  Therefore,  we  oonsider  the  case  a(s,t)  =  0  where  /?  is  typically  of  the  farm  /?(«)  =  1  —  crc. 
Assume  that  Ct  =  C(.,t)  is  a  C2 -classical  solution  on  some  interval  [O,*')  (<'  <  oo).  Thus  we  are 
considering  C2  solutions  of  the  system 


%  =  Ms,t))N 
C(st0)=Co{s), 


(1.3) 

(1.4) 


(Note  that  we  do  not  rule  out  the  possibility  that  a  C2 -solution  may  exist  for  all  t  >  0.) 
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Let, 


9(s>*)  :=  1^1  =  1 x\+fi\xri, 


denote  the  length  along  the  curve.  The  arc-length  parameter  5  is  then  defined  as 


s(s,t)  :=  [  g(£,t)d{. 

Let  the  positive  orientation  of  a  curve  be  defined  so  that  the  interior  is  to  the  left  when  traversing 
the  curve.  The  tangent,  curvature,  normal,  orientation  and  length  are  defined  in  the  standard 
way.  We  will  take  the  normal  to  be  pointing  outwards,  where  the  inward  or  outward  is  determined 
by  the  interior,  or  equivalently  by  the  orientation  of  the  curve.  We  then  have  that 


-* 

T:= 

k:= 

N:= 


dC  =  ldC 
ds  g  ds' 


1 dr  g'ds'' 
-g  -i  «f 
igl  *»  ds ' 


*:= 

L(t):=  I  g(s,t)ds 
Jo 


We  also  define  a  quantity  which  we  will  call  length-squared  by 

ZP(t)  :=  [  gl{s,t)ds. 

Jo 

Finally,  we  let 

«(<):=/  |<c(s,t)l^(s,t)<is, 

Jo 

denote  the  total  absolute  Gaussian  curvature. 


2.  On  General  Curve  Flow 


In  this  section  we  will  derive  the  evolution  equation  for  the  tangent  T,  normal  N,  metric  g, 
curvature  k,  orientation  9  and  length  L,  for  families  of  curves  satisfying  (1-1). 

It  is  easily  established  that 


which  will  be  used  in  the  following  proofs.  Note  that  these  evolution  equations  are  for  the  general 
deformation 

Moreover,  we  can  compute  that  the  metric  g  evolves  as  follows: 


9_  dC  dC 

dt~  ds'  ds> 
dC  d_dC 

~~2<  ds'dtds  > 

dC  d_dc 

-  <  ds'dsdt  > 

=  2<<1 (f,^-[«f  +  pN]> 

n  ~  da~  8T  30  -  adN 
-2<9T'  +  +  ds^  +  fids  > 

=  2  <  gf,a,T  -  augN  +  P,N  +  pngT  > 
=  2g[at  +  png]. 


Hence,  we  see  that 

^  =  q,  +  png. 

In  the  special  case  of  a  =  0, 

dt  ~ 

We  will  need  the  following  change  of  partials  for  computing  evolution  equations: 
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Next  we  have  the  following  evolution  equation  for  tangent: 


ST__  d_dC 
dt~  dtds 

-ir  ,R  JdC  d  dC 

=  —[a.  +  »k9\T  +  l[Qf  +  0ft] 

3 

=  —[a*  +  Pk9]T  +  a*T  +  ch-qZ  +  0tN  4-  P~q~ 

=  —[a,  +  fajft  +  -a.T  -  okN  +  -0$N  +  0k  f 
9  9  9 

=  -\0,  -  ang\N. 
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Similarly,  for  the  normal  we  see  that 

dN  dfi 

HT=<  ~aT'T  >  T 
at  at 

-4 '-»>? 

1. 

=  — [ft*  -  <*Kg\T. 

9 

Next  we  define  the  orientation  of  a  curve  as  the  angle  the  tangent  makes  with  the  x-axis.  Let 
f  =  (cos(0),sin(0)),  so  that  N  =  (sin(0),  —  cos(0)).  Then, 


|p(-sm(*),cos(0))^ 

=  JP- 


Therefore, 


As  for  curvature,  we  compute  that 


=  ir[&  “  aK9\ 


=  Kg. 


8k_  d_W_ 
dt~  dtds 

-It  .  .89  d.  80  ^ 
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-1.  .  .  &P  dn 

=  —  [a.+^g)K-w  +  alK  +  a^ 
8*0  Sk  .  , 

=  -W  +  a6l-^ 


For  the  length,  we  derive 


dL  8  f*  ,  t\A 

fr  =  ath 

=  >^ods 

[2* 

=  /  [a.  +  fag]  ds, 
Jo 


and  similarly  for  length-squared, 


8£<!>  9  [*,.  ... 

-ar=dih 

/*■  dg7(s,t)  , 


Jo  at 

fht 

=  /  2 g[a,  +  png]  ds. 

Jo 


We  now  specialize  to  the  case  0(k)  =  1  -  ck  which  is  a  common  model  frquently  used  in 
applications  such  as  flame  propagation,  crystal  growth,  among  others.  First,  we  can  easily  show 
in  this  case  that  the  metric  evolves  according  to 

|  =  (-«j + *),. 

Second,  for  the  tangent  and  normal  we  have 

?L  - 

dt  ~  * 

—  _  flif 

dt  "  g1' 

Next,  the  orientation  evolution  is  governed  by 
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Similarly,  one  can  show  that  the  evolution  equation  for  curvature  is 


Finally,  length  evolves  as 


9k  ,  3  0 

=  OC||  +  CK*  -  Kr . 


fix  . 

Lt  —2t  —  e  I  teg  ds. 
Jo 


It  is  also  useful  to  further  constrain  the  flow  and  obtain  evolution  equations  for  the  particular 
case  of  e  =  0,  or  /?(«)  =  1,  for  which  we  have 

dg 

at**9’ 

at  . 

aT=0' 

«L0, 

at  ' 

£=o, 

at 

9k  2 

at~ ~K  ’ 
at 


The  evolution  equation  for  curvature  may  be  solved  explicitly  as 

,  _  «M) 

1  +  k(s,  0)f  ’ 

This  implies  that  the  dassical  solution  will  fail  to  exist  when 


_  -1 
*  ~  k(s,  0)' 

Hence,  if  the  initial  curve  to  (1.3)  is  convex,  the  equation  will  have  a  dassical  solution  for  all 
time. 

The  metric  equation  may  also  be  solved  as 

~=  n(s,t)g(s,t) 

fh=  0) 

g  l  +  «(s,0)f 
fln  (g)  k(s,  0) 
dt  —  1  +  /c(s,0)t 
dln(p)  3ln(l  +  k(s,  0)i) 
dt  ~~  dt 

ln(ir(s,<))  -  In (0(s,O))=  ln(l  +  /t(s,0)<)  -  ln(l) 

ln(0(a,<))=  In  (0(5,0))  +  ln(l  +  n(s,0)t) 

0(5, 0=  0(5, 0)(!  +  rc(5,0)t) 


2.  ON  GENERAL  CURVE  FLOW 


7 


Hence,  the  metric  changes  linearly  in  time  with  a  curvature  dependent  coefficient.  In  particular, 
for  negative  curvature  the  metric  decrease  tc  zero  at  exactly  the  time  the  shock  is  formed. 
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3.  Bounds  on  Length,  Curvature,  and  Travelled  Distance 

In  this  section  we  give  bounds  on  the  length  and  total  absolute  curvature  for  the  family  defined 
in  Section  1.  As  before  we  are  particularly  interested  in  the  case  a  =  0  and  =  1  —  etc(s,t) 

which  is  of  interest  in  physical  applications  and  recently  in  computer  vision  [Kimia,  1989]. 

3.1  A  Bound  for  Length 

Lemma  3.1:  Let  C(s,t)  be  a  solution  of  (1.3)  for  t  €  [0,  <')  and  k/3(k)  <  M  for  all  k  E  R 
(regarding  p  as  a  function  of  k).  Then, 

L{t)<  min(L(0)  4-  Int,  L(Q)eMt). 

In  particular,  for  P{k)  =  1  —  ck, 

L(t)  <  min(L(0)  +  2ict, L(0)e&). 


Proof.  We  have 


So. 


L\  =  2ir  —  e 


Lt  <  2 ?r, 


L[t)  <  2 nt  +  L{ 0). 

Note,  the  equality  holds  for  e  =  0.  Alternatively, 


Since  k(3(k)  <  M 


dL  t** 

ar  =  i  ‘)K9ds 

dL  p 

ar-i  Mgd3 


<  ML(t) 


(In  [//(<)])'  <  M 


Therefore, 
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that  is, 

In  [£(*)]  <  Mt  +  ln(l(0)) 
L(t)  <  L(0)eMt. 

In  particular  for  P  =  1  -  ac,  M  =  £,  and 


L{t)  <  I( 0)e*. 


I 


Remarks  1. 

(i)  In  this  case,  as  e  -*  oo,  L(t)  <  L{0).  It  is  interesting  to  observe  that  the  two  estimates 
complement  each  other:  for  small  c  the  second  estimate  is  very  large  making  the  first  estimate 
more  useful.  However,  for  large  e,  the  first  estimate  is  exaggerated  making  the  second  estimate 
more  useful. 

(ii)  The  same  proof  as  above  shows  that  the  length-squared 

I<2>(<)  :=  Pfds 

J  0 

is  bounded  by  Z/2)(0)e2M‘.  (This  fact  oould  also  have  been  used  to  bound  L(t),  but  the  bound 
would  have  been  more  conservative.) 

3.2  A  Bound  for  Total  Absolute  Curvature 

Lemma  3.2:  Let  C(s,t)  be  a  solution  of  (1.3)  for  t  €  [0,  t').  Suppose  that  k/3(k)  <  M,  and 
At  <  0  Then, 


R(t)  <  R(0). 


Proof.  Define 

q(t):=jf  q{K{a,t))g{s,t)ds, 

where  q  is  the  piecewise  smooth  convex  approximation  of  f(x)  =  |x|  given  by 


i*i 

i  +  f*2 


if  x 
if  x 


>  I 

<1 

—  n 


Then 

fix  fix 

tf(t)=  Jo  9n(A)rctSf  ds  +  q{K)gtds 

-  ~  JQ  ?*(*)(A«  +  P*2)9  ds  +  J  M(P«9)  ds 

rL(t)  fix 

=  -  /  +  /  b(«)  -  [P*g)  ds. 

JO  JO 

=  -(9*(k)Ai)  1?°  +  f*  ]  d*+  Jo  [?(«)  -  «?*(«)] {P*g)  ds. 
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fH  t)  ft* 

=  qH*(K)Kl0lds+  \q(K)- KqK(K)]{Png)ds. 

Jo  Jo 

fL(t)  t7* 

=  /  (*)«!&<**+ /  [q(n)  -  KqK(K)](PKg)ds. 

Jo  Jo 

t 

Since  /?„  <  0  and  convexity  of  q  requires  q"  >  0,  we  have 

/  fe(*)  -  KqK(K)](Png)  ds. 

Jo 

Note  that 

*(t)  <  m , 

so  that  a  bound  on  q  is  a  bound  on  k.  Moreover,  we  have  that 

0  <  $(*)  -  xq'{x)  <  {  j_  \Xx\l 

V  2n  fi 

Now  since 

P{k)k  <  M, 

and  fe(/e)  -  «?*(«)]  >  0, 

#{i)<  M  I  [?(«)  -  /cq*(«)]$r  ds. 

JO 

< 

and  so, 

*'(!)< 

Since  by  lemma  3.1  the  length  is  finite,  letting  n  — >  oo,  we  get 

R'(t)  <  0, 

or 

R(t)  <  3(0). 

I 

Remark  2.  The  conditions  of  the  above  lemma  holds  for  P(K(s,t))  =  1  -  oc(s,<),  for  e  >  0. 
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Lemma  3.3:  Let  a  family  of  curves  satisfy  (1.3)  with  convex  initial  condition,  i.e.  k(s,  0)  >  0. 
Then,  R{t)  —  k{ 0)  =  2ir  and  the  curve  remains  convex  for  all  times. 

Proof.  For  convex  curves, 


0)  =  /  xg  da  =  2x. 
Jo 


There  is  some  neighborhood  of  time  such  that  x{a,t)  >  0  for  all  a.  Therefore, 

die  d  ft* 


ok  dt**  , 

aTa/.  ** 

-  /  [Ktg  +  Kgt]da 
Jo 

=  /  [{-pit- P*7)g  +  KpKg]ds 
JO 

[** 

=  -/  Pu9da 

Jo 

rW)  n 

=  -  /  Pn  da 

Jo 


=  -A 
=  o. 


Hence, 


S(<)  =  2tt. 


Since  only  oonvex  curves  can  satisfy  this  condition,  the  evolved  curve  must  be  convex.  | 


Lemma  3.4:  Let  a  family  of  curves  satisfy  (1.3)  for  which  /?*  <  0.  Then,  if  ks (s,t)  ^  0  for 
all  a  and  0  <  t 

*(t)  <  S(0) 


.  For  pK  =  0, 


R(t)  =  «(0). 


Proof.  This  proof  is  due  to  Sethian  [Sethian,  1985]  and  we  include  it  here  for  completeness  with 
a  few  changes. 

Without  loss  of  generality  pick  the  starting  point  I  =  0  to  be  a  zero  of  curvature  such  that 
positive  curvature  begins  in  the  positive  direction  of  the  curve.  Partition  the  interval  [0,L(f)] 
into  n  +  1  maximal  subintervals, 

{[So  =  0>  $i]j  (5ljS2]> "  ' '  il^ij^i+l]*  *  *  ‘  =  L{t)] }> 

such  that  k{5)  is  entirely  positive,  negative,  or  zero  in  the  interval  (S,,3,+i)  for  t  =  1,2, •••  ,n. 
Then,  S*  are  zeros  of  curvature,  ic(5,)  =  0.  Note  further  that  in  general  =  Sj(t).  Also,  let 

{1  if  k(S)  >  0  for  S  €  (o,  6) 

0  if  k(3)  =  0  for  a  €  (a,  b) 

-1  »f  k(3)  <  0  for  S  €  (a,  5) 
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Then, 


d 

1* , 

at 

Jo  1 

d< 

f  r 

«; 

n 

1 

i=0 

J* 

ft 

y' 

1 

•=0 

.dSf+l  V~'i .  /  _  .\i  /  .\&si 


Since  intervals  for  which  curvature  is  uniformly  zero  do  not  contribute  to  the  sum,  we  will  discount 
them.  Without  loss  of  generality  assume  otherwise  in  the  following. 

= yi  r+i  iA(- a*  -  ^2) + wk]  d 5 

i=0Jt*  1*1 

=  P*1  \p(KA5iisi+i)])P"]ds 

=  |t*‘ 


=  -B’talA’A+iJDIAP.+i)  -  A(S i)) 

i=0 

Now,  by  our  original  assumption  p(n,  [So,  ’Si])  =  1-  The  next  interval,  then,  has  negative  curvature 
and  p(k,[Si,S3])  =  —1.  Since  zeros  of  curvature  must  pair  up,  p(/c,[S„,Sn+i])  =  -1.  In  short, 
p(ic,  [Si, a, +ij)  =  (-1)*.  Therefore, 

§=-B- 

isO 

=  -  B-iyA(*«) + D-iy  A(so 

vO  «=0 

n+1  w 

=E(-iyA(*i)+D-iyA(A) 

j=l  **0 

=B-iyAft)+t(-iyA(A) 

jaO  nO 

=  2B-1)'A(I.) 
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=  2B— WfrMSi) 

tsO 

Now,  since  has  sign  (—1)'  if  <  0,  then 


so  that 

k (t)  <  «(0). 

However,  if  f}K  =  0,  such  as  the  case  with  /5  =  1, 

*(<)  =  «(0). 

I 

Remark  3. 

In  conclusion,  convex  curves  remain  convex  and  H(t)  =  2ir  for  all  deformations.  For  nonconvex 
curves  and  /?  =  1,  we  have  £(<)  =  k(0).  Note  therefore  that  for  all  curves,  the  deformation  /?  =  1 
does  not  alter  the  total  absolute  curvature.  Finally,  for  nonconvex  curves  and  deformations  for 
which  fiK  <  0,  such  as  fi  =  1  —  ck  with  e  >  0,  we  have 

H(t)  <  Jc(0). 

This  describes  the  important  role  of  e  in  the  deformation  as  one  of  reducing  the  total  absolute 
curvature.  Note  that  for  =  — e«,  the  deformation  will  evolve  an  embedded  curve  to  a  circle 
[Gage  and  Hamilton,  1986;  Grayson,  1987]. 
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4.  How  Far  Can  a  Curve  Travel? 

In  this  section  we  will  address  the  key  issue  of  how  far  the  evolved  curve  can  be  away  from  the 
initial  curve.  First  we  derive  a  relationship  between  the  distance  of  a  point  from  a  curve  and 
the  curvature  of  the  curve  at  the  nearest  point.  Second,  the  rate  of  change  of  distance  of  a 
point  to  a  curve  is  related  to  tire  speed  of  the  curve  at  its  nearest  point.  This  result  holds  for 
nonshock  points.  Third,  the  distance  of  a  point  from  a  curve  bounds  the  rate  of  change  of  that 
distance  with  time.  Fourth,  we  show  that  a  curve  can  not  travel  too  far  pointwise.  Fifth,  for 
any  time  neighbourhood  for  which  the  curve  does  not  travel  beyond  e,  we  constrain  its  expansion 
as  a  function  of  time.  From  these  we  conclude  that  two  curves  dose  in  time  are  dose  in  their 
Hausdorff  distance.  Finally,  a  theorem  shows  that  the  limit  of  curvature  evolution  exists,  and 
using  the  above  we  can  even  bound  the  total  Gaussian  curvature. 

4.1  On  the  Distance  Travelled 

In  what  follows  we  will  limit  ourselves  to  the  case 


/3(#c)  =  1  —  ac. 

For  a  subset  S  C  R2.  let  Ns(S)  denote  a  dosed  6— neighborhood.  Define  the  signed  distance 
of  a  point  from  a  curve  (regarded  as  a  point  set  in  R2)  as 


{inf  {d(p,  <7)|?  €  Ct]  if  p  is  outside  Ct 
-  inf  {d(p,  g)|g  €  Ct }  otherwise, 


where  outside  is  the  region  to  the  right  of  the  curve  as  one  traverses  the  curve  in  the  positive 
orientation.  In  this  section  we  will  consider  an  arbitrary  point  in  the  plane,  p,  and  consider  its 
relation  to  the  curve  Ct.  Set 

d(t)  :=  d(p,C<) 


Lemma  4.1:  Let  p^C  be  a  point  in  R2.  Let  q  be  the  closest  point  on  the  curve  to  p.  (Note 
that  q  exists  by  compactness.)  Then, 

j  *(9)  >  if  p  is  outside  C 

\  *(9)  <  if  Pis  inside  C. 

Proof.  Set  d  =  dfj>,q).  First  suppose  the  point  p  is  outside  the  curve,  so  that  d  >  0.  Let  q 
be  the  dosest  poin-  on  the  curve  C  to  p.  Consider  the  drde  of  radius  d  and  center  p  which  is 
tangent  to  the  curve  at  q.  We  have  two  separate  cases:  (i)  the  curve  has  non-negative  curvature 
at  q.  In  this  case,  x(q)  >  =£,  trivially,  (ii)  the  curve  has  negative  curvature  at  q.  In  this  case,  the 
curve  C  fies  entirely  outside  the  drde.  In  order  to  see  this  suppose  to  the  contrary;  then  there 
exist  points  on  C  closer  to  p  than  d,  a  contradiction.  Therefore,  the  curvature  of  the  drde  3  is 
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greater  than  the  curvature  of  the  curve  at  q,  i.e.  J  >  — ic(q).  Therefore,  for  both  cases  when  the 
point  p  is  outside  the  curve  C,  we  have 

Now,  as  required  suppose  the  point  p  is  inside  the  curve  C,  d  <  0,  and  q  the  dosest  point 
of  the  curve  C  to  it.  Again,  consider  the  arde  of  radius  —  d  and  center  p  which  is  tangent  to 
the  curve  at  q.  Once  more  we  have  two  cases:  (i)  the  curve  has  non-positive  curvature  at  q. 
curvature.  Trivially,  then  #c(q)  <  (i)  the  curve  has  positive  curvature  at  q.  In  this  case,  the 

arde  again  lies  entirely  within  the  curve,  touching  it  only  at  q,  so  that,  the  curvature  of  the  arde 
is  greater  than  the  curvature  of  the  curve  ic(q).  Then, 

*(q)  <~-  | 


Lemma  4.2:  Let  p  be  a  point  in  R2  where  Ct  deforms  along  the  normal  according  to  (1.3). 
If  can  be  bounded,  then 

9 

=  'I' 

Proof.  Let  q(s,t)  be  the  dosest  point  to  p  on  Ct  and  q(s  +  6s,  t  +  6t)  the  dosest  point  on 
C(.,t  +  St)  to  p.  Since  the  line  (p,  q(s,t))  is  normal  to  Ct,  the  point  q(s,t  +  St)  is  on  this  line 
a  distance  St  from  q(s,t).  Consider  the  triangle  with  vertices  p,q(s,t  +  St),q{s  +  Ss,t  +  St), 
where  the  angle  at  p  is  denoted  by  60.  Then, 


+  «)-««)  i  tW)  -  gft)  _ 
St  ~  St1  cos  {SO) 


=  -2<f(() 


ec 

at 


cos  (SO)St  cos(SO) ' 


In  the  Emit,  St  — ►  0,  SO 
Section  2, 


Since  0t  can  be  bounded 


I 


0.  From  the  equations  for  rate  of  change  of  orientation  with  sin 

<*  =  dh. 

dt  g 


Remark  4.  Observe  that  as  t  aproaches  die  time  of  shock  formation,  g(s,t)  goes  to  zero,  and 
therefore  the  condition  of  Lemma  4.2  do  not  hold  in  the  limit.  - 

Lemma  4.3:  Let  Ct  be  a  solution  of  (1.3)  inhere  0(k)  =  1  -  at.  Let  p  be  a  point  in  R2.  If 
d(t)  =  d(p,Ct)<e,  then, 


<f(t)d(t)  >  -2c. 
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Proof.  Let  q(t)  be  the  closest  point  on  the  curve  Ct  to  p.  First,  consider  the  case  where  the 
point  p  is  outside  the  curve  Ct.  Then,  by  Lemma  4.1,  *(9(t))  ^  Consequently, 


<f(t)=  at  —  1 

>  zb 

-  d(ty 


since  d(t)  is  positive  and  d(t)  <  t.  Hence  we  can  conclude  that 

<f(t)d(t)  >  -2c. 

Now,  consider  the  case  where  the  point  p  is  inside  the  curve  which  implies  n(q(t))  <  ^jj.  Then, 

(f(t)=  at  —  1 


since  d(t )  is  negative  and  d(t)  <  t.  Once  again,  we  can  conclude  that 

(f(t)d(t)  >  -2c. 


I 

Lemma  4.4:  Consider  a  curve  Co  evolving  through  a  function  of  curvature  as  in  (1.3),  with 
0(k)  =  1  —  at.  Then  for  each  t,  there  exists  t  =  t(c,C0)  >  0,  such  that  for  each  p  €  Ct*  with 
0  <  t  <  t  we  have  that 

d(t)  <  e  VO  <t<t. 

Proof.  Let  p  €  Cf  for  some  0  <  f  <  t .  Note  that  1  —  ot(q{t),  t)  is  the  speed  of  the  point  q{t) 
on  Ct  in  the  evolving  family.  Now  since  *(*,<)  is  a  periodic  solution  of  a  polynomial  reaction- 
diffusion  equation  with  analytic  coefficients  and  smooth  initial  condition,  there  exists  an  interval 
[0,ti]  such  that  rc(a,f)  is  uniformly  bounded  as  a  function  in  t  say  by  Af.  Therefore  on  [0,tij 

|1  -  oc(q(t),f)l  <  1  + 

Thus  any  point  q(0)  €  Co  cannot  have  travelled  more  than  distance  c  from  Co  in  time 

<2  :=  1  +  tM' 


Set 


f  :=  min(ti,t2). 
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Since  p  €  C{  for  some  0  <  ?  <  f,  the  lemma  is  proven.  | 

Theorem  4.5:  Consider  a  curve  Co  evolving  through  a  function  of  curvature  as  in  1.3  with 
P(k)  =  1  -ck.  Let  t  =  t(e,Co)  be  as  in  the  previous  lemma.  Then 

Ct  c  N>/tt(^o)> 


for  all  t  €  [0,  <]. 

Proof.  jFrom  Lemma  4.4,  given  e  there  exists  t  such  that  for  all  t  €  [0,  <]  and  for  all  p  €  Ct  we 
have 

<nt)  =  dWA)  <  t- 

Now,  by  Lemma  4.3, 

i^r = *m*)  > 

for  all  t  €  [0,i\.  By  integration 

d(t)  <  y/to. 

Hence,  we  have  shown  there  is  t  such  that  for  t  €  [0,  i]  we  have 

Ct  C  N^(Co). 


I 

Remark  5.  For  djj  the  Hausdorff  metric  defined  on  compact  subsets  of  R2,  from  the  above 

theorem  we  have  that  _ 

dH{Ct,Co)  <  Vtet. 


4.2  Limits  of  Classical  Solutions 


Lemma  4.6:  Let  Ct  :  S1  — *  R2  be  a  family  of  C2  functions  with  uniformly  bounded  length- 
squared  L^(t).  Then,  Ct  is  uniformly  equicontinuous. 

lemma  is  a  simple  modification  of  the  result  for  functions  with  bounded  derivatives. 
C(S,()-CK,!)  = 

L  is  the  uniform  bound  on  the  length-squared 


Proof.  The 
Note  that 

Therefore  if 
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‘Lc(!)'*@y 


1/7 


do\  <  L\s  —  s0^2, 


that  is,  the  family  is  equicontinuous  with  Holder  constant  L  and  exponent  1/2.  | 


Theorem  4.7:  Consider  a  curve  Co  evolving  through  a  function  of  curvature  as  in  (1.3). 
Then, 

HmCt  =C*, 

t->t' 

in  the  Hausdorff  metric.  The  curve  C*  regarded  as  a  mapping C*  :  S1  — ►  R2  is  Holder  continuous 
with  exponent  1/2. 

Proof.  Since  the  lengths-squared  of  the  curves  Ct  are  uniformly  bounded  (see  Lemma  3.1  and 
Remark  l(ii))  regarding  each  Ct  :  Sl  — ►  R2,  we  can  apply  the  Lemma  4.6  to  the  family 

{&}«€{<>,<'] 

to  conclude  that  it  is  equicontinuous.  Moreover  from  Theorem  4.5,  the  curves  lie  in  a  compact 
region.  Thus  by  the  Arzela-Ascoli  theorem  and  the  proof  of  Lemma  4.6,  there  exists  a  uniformly 
convergent  subsequence  Ctn  — ►  C*.  where  C*  :  S1  — ►  R2  is  a  Holder  continuous  function  with 
exponent  1/2.  (The  Holder  continuity  of  the  limit  follows  from  the  fact  that  the  family  is 
equicontinuous  and  Holder  continuous.  C*  will  also  denote  the  corresponding  curve.)  Thus  as 
compact  subsets  of  the  plane,  we  have  that  Ctn  -*  C*  in  the  Hausdorff  metric. 

Ct  — ►  C*  (in  the  Hausdorff  metric)  as 


Note  that  for  all  t  6  we  have 


Therefore,  by  Theorem  4.5 

C,CNsP&{Cu')’ 

Ct  C  N{/2(Ct„), 

Ct  C  NS(C*), 

as  required.  | 

Remarks  5. 


To  complete  the  proof,  we  need  to  show  that  all  the 
t  —*tf.  Let  8  >  0,  and  choose  4  such  that 


and 


Ctn  c  n5/2(c*) 


(We  choose  6  >  0  sufficiently  small  so  that  H  —  ^  >  0.) 


*~tn<  16e' 


4.  HOW  FAR  CAN  A  CURVE  TRAVEL? 


19 


(i)  Note  that  since  C*  is  Holder  continuous,  and  since  the  total  Gaussian  curvature  of  the  family 
is  uniformly  bounded  (see  Lemma  3.2.1),  C*  will  have  finite  total  Gaussian  curvature. 

(ii)  From  the  above  results,  we  have  a  fairly  complete  picture  about  the  the  classical  evolution  of 
a  family  of  curves  with 

P(k)  =  1  —  6*  c  >  0. 

In  a  future  paper,  we  would  like  to  study  weak  solutions  of  this  family  (after  possible  shock 
formation)  with  an  emphasis  on  the  physical  applications  to  vision. 
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Potentials,  Valleys,  and  Dynamic  Global  Coverings1 
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Abstract 


We  present  a  new  approach  to  effect  the  transition  between  local  and  global  represen¬ 
tations.  It  is  based  on  the  notion  of  a  covering,  or  a  collection  of  objects  whose  union  is 
equivalent  to  the  full  one.  The  mathematics  of  computing  global  coverings  are  developed 
in  the  context  of  curve  detection,  where  an  intermediate' representation  (the  tangent  field; 
provides  a  reliable  local  description  of  curve  structure.  This  local  information  is  put  together 
globally  in  the  form  of  a  potential  distribution.  The  elements  of  the  covering  are  then  short 
curves,  each  of  which  evolves  in  parallel  to  seek  the  valleys  of  the  potential  distribution.  The 
initial  curve  positions  are  also  derived  from  the  tangent  field,  and  their  evolution  is  governed 
by  variational  principles.  When  stationary  configurations  are  achieved,  the  global  dynamic 
covering  is  defined  by  the  union  of  the  local  dynamic  curves. 

1.  Introduction 


One  of  the  key  questions  in  computational  vision  is  how  to  effect  the  transition  from 
local  representations  to  global  ones.  For  example,  edge  operators  give  an  indication  of  local 
edge  position  and  orientation;  how  can  they  be  threaded  together  into  a  global  contour? 
A  popular  solution  is  commonly  called  “edge  following”  or  contour  tracing  [Ballard  and 
Brown,  1982;  Levine,  1985]  and  it  works  as  follows:  from  a  starting  edge  position,  move 
in  the  direction  indicated  by  the  orientation  until  the  next  edge  point  is  encountered,  then 
move  in  that  direction,  and  so  on  until  the  final  point  is  reached.  But  such  algorithms  are 
inherently  sequential,  and  result-in  myriad  “garden  paths”  if  any  errors  or  noise  are  present. 
Global  parameters,  such  as  starting  and  ending  points;  total  number  of  edge  points;  or 
total  contour  length,  aid  in  their  application,  but  are  rarely  available  (except  through  user 
interaction,  e.g.,  Kass,  Terzopoulos,  and  Witkin,  [1987]).  To  make  matters  worse,  inferring 
global  parameters,  such  as  the  total  length  of  a  curve,  raises  problems  as  difficult  as  global 
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curve  detection  itself!  Other  methods,  more  sophisticated  than  simple  edge  following,  been 
developed  in  an  attempt  to  apply  global  constraints,  and  include  minimizing  properties 
such  as  contrast  variation  along  the  curve,  total  curvature  [Montanan,  1971:  Martelli,  1974]. 
smoothness  across  scale  space  [Lowe.  1988].  energy  within  plate  and  membrane  models  [Blake 
and  Zisserman.  1987:  Terzopoulos.  1986].  or  applying  top  down  information  [Tsotsos.  1987 
;  Draper  tt  al.  1989].  However,  few  of  the  objects  in  our  visual  world  are  made  from  bent 
plates,  and  generally  problems  arise  in  verifying  when  such  constraints  are  valid  globally.  It 
is  almost  as  if  the  cart  is  being  put  before  the  horse,  in  that  (hypothetical)  global  information 
is  being  used  to  force  local  decisions. 

In  an  attempt  to  turn  this  situation  around,  we  introduce  a  new  class  of  algorithms 
for  synthesizing  contours  from  local  representations  of  their  differential  structure.  These 
algorithms  are  inherently  parallel  and  are  natural  for  vision  applications.  The  mathematical 
idea  behind  these  new  algorithms  is  to  compute  properties  of  global  structures  indirectly 
by  computing  a  covering  of  them.  Loosely  speaking,  a  covering  of  an  object  is  a  collection 
of  different  objects  whose  union  is  equivalent  to  it.  In  our  case,  we  shall  define  a  covering 
of  a  global  curve  that  consists  in  a  collection  of  short,  overlapping  curves.  Each  of  these 
derives  from  a  local  representation  of  image  information  (which  we  call  a  tangent  field)  and 
then  evolves  like  the  “snakes”  of  Kass  et.  al  [1988]  according  to  a  potential  distribution  also 
derived  from  the  tangent  field.  But  unlike  standard  “snakes”,  ours  are  local  objects;  it  is 
their  union  that  comprises  the  cover.  Each  element  of  the  cover  evolves  dynamically  and  in 
parallel,  and  no  external  global  information  is  required.  It  is  all  available  by  construction  in 
the  potential  distribution.  For  these  reasons  we  refer  to  our  representation  of  global  curves 
as  dynamic  coverings. 

The  idea  of  computing  dynamic  coverings  developed  in  a  biologically-oriented  project  of 
curve  detection,  a  brief  overview  of  which  is  provided  next  (see  also  [Zucker,  Dobbins  and 
Iverson,  1989].  Within  this  context,  the  key  to  computing  dynamic  coverings  is  to  impose 
two  representations  between  the  image  and  global  curves:  a  tangent  field  and  a  potential 
distribution..  The  tangent  field  (defined  in  Sec.  2)  represents  the  local  properties  reliably,  and 
the  potential  distribution  puts  them  together  globally.  These  intermediate  representations 
thus  break  the  enormous  gulf  between  images  and  global  contours  into  more  manageable 
“chunks”,  and  represent  another  key  difference  between  our  approach  and  other  recent  ones 
[Kass  et  al,  1987;  Blake  and  Zisserman,  1987].  We  infer  the  tangent  field  from  the  image,  and 
then  build  the  potential  distribution  and  snakes  from  it;  other  researchers  attempt  to  build 
the  potential  distribution  directly  from  the  image.  We  can  actually  calculate  the  generators 
for  the  potential  distribution  from  the  equivalence  class  of  curves  that  map  into  each  tangent 
field  entry. 

The  paper  is  organized  as  follows.  Following  the  overview  of  our  approach  to  curve 
detection  (Sec.  2),  we  develop  the  preliminary  notions  of  potential  distributions  (Sec.  3) 
and  deformable  curves  (Sec.  4).  We  then  get  to  the  heart  of  the  paper  in  Sec.  5,  where  the 
precise  definition  of  a  global  covering  is  developed.  Examples  of  the  dynamic  computation 
of  global  coverings  are  finally  shown  in  Sec.  6,  for  both  artificial  and  natural  images. 
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2.  Overview  of  the  algorithm 

The  algorithm  for  curve  detection  consists  of  two  distinct  stages,  the  first  leading  to  a 
coarse  description  of  the  local  structure  of  curves,  and  the  second  to  a  much  finer,  global 
one  [Zucker.  David,  Dobbins,  and  Iverson.  1988], 


2.1  Stage  1:  Inferring  the  tangent  field 

Orientation  selection  is  the  inference  of  a  local  description  of  a  curve  everywhere  along  it. 
Formally,  this  amounts  to  inferring  the  trace  of  the  curve,  or  the  set  of  points  (in  the  image) 
through  which  the  curve  passes,  its  quantized  tangent  and  curvature  at  those  points,  and 
the  discontinuities  [Zucker,  1985].  Through  all  this  work,  we  will  refer  to  such  information 
as  the  tangent  field. 

This  first  stage  of  orientation  selection  is  in  turn  modeled  as  a  two  step  process: 


Step  1.1.  Initial  Measurement  of  the  local  fit  at  each  point  to  a  model  of  orientation  and 
curvature.  The  form  of  the  initial  measurements  is  biologically  motivated,  and  a  model 
of  endstopped  simple  cells,  which  encode  both  the  orientation  and  curvature  estimates 
[Dobbins,  Zucker,  and  Cynader,  1987]  is  used  to  perform  them.  The  local  measure¬ 
ments  are  quantized  into  discrete  classes  of  orientation  and  curvature.  However,  since 
the  local  measurements  are  inherently  inaccurate,  we  require 


Step  1.2.  Interpretation  of  the  local  measurements  such  that  they  become  globally  con¬ 
sistent.  Curvature,  which  relates  neighboring  tangents,  is  used  to  define  a  natural 
functional  to  be  minimized  in  order  to  attain  consistency  [Parent  and  Zucker,  1989; 
Iverson  and  Zucker,  1988].  Relaxation  labelling  [Hummel  and  Zucker,  1983]  provides 
the  formal  framework  for  this. 


A  complete  treatment  of  this  first  stage  can  be  found  in  Iverson  [1988]. 

At  the  end  of  the  first  stage,  the  tangent  field  gives  the  discrete  trace  of  the  curves  in  an 
image.  The  two  steps  above  are  necessary  to  guarantee  that  it  contains  reliable  information. 
More  precisely,  the  tangent  field  is  a  set  of  n  3-tuples, 

{((*i.w)»*i»«i)  I  (*i.y«)  €  I,0i  €  0,/c,-  €  AC} 

*  =  l,...,n,  where  n  is  the  number  of  trace  points  in  the  image  7,  (xi,y,)  denotes  the 
(quantized)  grid  coordinate  of  the  ith  trace  point  indexed  over  the  image  7,  and  0,  and 
«,  are  its  quantized  orientation  and  curvature,  respectively.  Typically,  0  consists  of  eight 
orientation  classes,  and  AC  of  five  curvature  classes  (see  Fig.  1(a)).  Thus  the  tangent  field 
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summarizes  the  local  differential  properties  of  curves  in  what  Zucker  [1987]  described  as  a 
qualitative  manner. 

Two  topological  properties  of  the  tangent  field  are  important:  (1)  each  curve  in  the 
image  is  represented  as  a  connected  (in  the  discrete  sense)  set  of  tangent  field  entries:  and 
(2)  discontinuities,  intersections  and  bifurcations  are  represented  as  multiple  tangent  field 
entries  at  identical  coordinates.  To  elaborate  on  (1).  recall  that,  except  on  the  border,  each 
point  (x,y)  of  a  (digital)  image  has  four  horizontal  and  vertical  neighbors,  and  four  diagonal 
neighbors:  these  eight  points  are  called  the  (3x3)  neighborhood  of  (x,y).  A  chain  between 
two  points  (xi,  j/i)  and  (X2,y2)  iS  a  sequence  of  points  where  the  first  one  is  (xi,yi),  the  last 
one  is  (x 2,y2).  and  each  point  of  the  sequence  is  in  the  neighborhood  of  the  preceding  one. 
A  set  of  points  of  an  image  is  then  sand  to  be  connected  when  there  exists  a  chain  between 
any  two  points  of  the  set  [Rosenfeld  and  Kak,  1982].  For  an  elaboration  of  (2),  see  [Zucker 
et  al.,  1988], 

2.2  Stage  2:  Inferring  a  covering  of  the  curve 


Since  the  tangent  is  the  first  derivative  of  a  curve  (with  respect  to  arc  length),  the  global 
curve  can  be  recovered  as  an  integral  through  the  tangent  field.  Such  a  view  typically  leads  to 
sequential  recovery  (numerical  integration)  algorithms. (e.g.,  contour  following).  But  these 
algorithms  require  starting  points  and  some  amount  of  topological  structure  (i.e.,  which 
tangent  point  follows  which),  and  there  are  immense  difficulties  inherent  in  obtaining  these 
parameters  directly  from  the  image.  Moreover,  because  we  are  interested  in  biologically 
plausible  models,  we  shall  require  that  all  the  algorithms  be  parallel. 

The  key  idea  behind  our  approach  is  to  recover  the  global  curve  by  computing  a  covering 
that  is  sufficent  for  obtaining  any  global  properties  of  the  curve  required  for  subsequent 
visual  processing.  The  elements  of  the  covering  are  short  curves,  initially  born  at  each 
(discrete)  tangent  location  with  unit  length,  but  which  then  evolve  according  to  a  potential 
distribution  constructed  from  the  tangent  field.  Thus  the  algorithm  that  we  are  proposing 
can  be  viewed,  mathematically,  as  approximating  the  computation  of  integral  curves  through 
a  direction  field  in  a  parallel  manner  with  very  non-standard  initial  conditions  (provided,  of 
course,  that  multiple  entries  at  the  same  position  are  properly  interpreted). 

Again,  there  are  two  conceptually  distinct  steps  to  Stage  2  of  the  algorithm: 


Step  2.1.  Constructing  the  Potential  Distribution  from  the  discrete  tangent  field.  Assum¬ 
ing  that  each  tangent  in  the  tangent  field  gives  a  coarse  hypothesis  about  the  behavior 
of  the  curve  in  its  neighborhood,  for  each  tangent  field  entry  we  represent  this  class 
of  hypotheses  by  an  elongated  (in  the  tangent  direction)  2D-Gaussian.  This  weight¬ 
ing  function  indicates  the  (equivalence)  class  of  curves  which  project  to  each  tangent 
field  entry  in  extending  the  discrete  tangent  in  both  length  and  width.  The  poten¬ 
tial  distribution  is  the  pointwise  summation  of  these  n  (one  per  tangent  field  entry) 
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Figure  1  Illustration  of  the  two  intermediate  structures  between  the  image  and 
the  global  curves:  the  tangent  Held  and  the  potential  distribution.  The  image  is 
a  section  of  a  fingerprint  image,  where  each  individual  pixel  is  shown  as  a  little 
block,  (a)  The  tangent  field,  superimposed  onto  the  original  image.  Each  tangent 
is  represented  as  a  unit  length  straight  line  in  the  tangent  direction  (curvature  is 
not  displayed),  (b)  The  potential  distribution,  computed  from  the  discrete  tangent 
field  by  pointwise  summation  of  elongated  20  Gaussians.  Notice  the  presence  of 
smooth  valleys,  corresponding  to  the  curves  in  the  image. 
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Gaussians.  The  key  feature  of  the  potential  distribution  is  the  presence  of  smooth 
valleys,  constructed  by  summation  of  nearby  Gaussians.  These  valleys  indicate,  with 
theoretical  continuum  precision  (although  the  implementation  is  limited  to  machine 
precision),  the  expected  positions  of  the  curves  in  the  picture  (see  Fig.  1(b)).  The 
construction  of  the  potential  distribution  thus  effects  a  transition  from  a  local  to  a 
global  representation  of  structure  and  shape. 


Step  2.2.  Curve  Dynamics.  The  valleys  of  the  potential  distribution  are  located  by  dy¬ 
namic  unit-length  curves,  born  at  each  tangent  position  in  the  tangent  field.  They 
evolve  according  to  a  variational  scheme  that  depends  on  curve  properties  (tension 
and  rigidity)  as  well  as  on  the  potential  distribution.  The  evolution  takes  two  forms: 
(i)  a  migration  in  position  to  achieve  smooth  coverings  at  subpixel  accuracy;  and  (ii) 
a  growth  in  length  such  that  nearby  covering  elements  overlap;  see  Fig.  2.  At  the  end 
of  the  migration  process,  we  shall  have  that  each  valley  is  completely  covered  by  this 
collection  of  curves,  and  each  curve  will  have  deformed  itself  to  fit  the  exact  shape  of 
the  valley.  We  call  this  collection  of  curves  the  global  covering  of  the  curves  in  the 
image. 


The  dynamic  curves  of  the  second  stage  are  a  generalization  of  “snakes”  [Kass,  Witkin 
and  Terzopoulos,  1988].  The  key  differences  between  our  approach  and  theirs  are  twofold: 
(i)  our  scheme  is  parallel,  so  questions  about  total  length,  boundary  points,  etc.,  are  handled 
implicitly;  and  (ii)  our  scheme  is  constructive,  with  the  potential  distribution  computed 
directly  from  the  tangent  field,  so  questions  about  features,  degree  of  image  blurring,  etc., 
axe  handled  explicitly.  We  must  neither  have  a  user  supply  information  interactively,  as  e.g. 
Kass  et  al.  were  required  to  do,  nor,  to  avoid  smoothing  across  discontinuities,  do  we  have 
to  distribute  penalties  globally,  as  e.g.  Blake  and  Zisserman  were  required  to  do.  Imposing 
stable  intermediate  structures — the  tangent  field  and  the  potential  distribution  (see  Fig. 
1) — between  the  image  and  the  global  curves  was  the  key,  and  the  mathematical  notion  of 
coverings  provided  the  unifying  structure. 

We  now  proceed  to  develop  the  notions  of  potential  distribution  and  dynamic  curves, 
before  putting  them  together  into  global  dynamic  coverings  in  Sec.  5. 


3.  The  potential  distribution 

The  local  descriptors  in  the  tangent  field  are  synthesized  into  a  global  description  rep¬ 
resented  as  a  potential  distribution.  In  this  Section  we  first  articulate  this  construction 
process,  then  define  a  mathematical  notion  of  a  valley  in  the  potential  distribution.  Since 
valleys  are  intended  to  be  the  locus  of  points  that  represents  (the  trace  of)  a  curve,  finally 
we  show  that,  for  simple  potential  distributions,  these  valleys  do  indeed  correspond  to  the 
global  curves. 
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Figure  2  Illustration  of  how  the  collection  of  short  dynamic  curves  evolve  to  cover 
the  locus  of  points  comprising  the  valleys  in  the  potential  distribution.  In  the 
usual  order,  the  covering  elements  migrate  from  their  initial  positions  (top  left)  to 
reach  the  bottom  of  the  valley  indicating  the  edge  (bottom).  Note  the  two  aspects 
of  the  movement:  a  migration,  and  a  length  increase. 
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3.1  Constructing  the  Potential  Distribution 


The  tangent  field,  consisting  of  discrete  trace,  discrete  tangent,  and  discrete  curvature, 
gives,  at  each  trace  point,  a  coarse  estimate  of  the  local  behavior  of  curves.  Since  each 
tangent  field  entry  corresponds  not  to  a  single  curve  in  the  world,  but  rather  represents 
the  equivalence  class  of  possible  curves  that  would  project  onto  this  tangent  field  entry,  we 
represent  this  (local)  equivalence  class  of  possibilities  by  the  (Gaussian-weighted)  Wiener 
measure  over  them.  The  Wiener  measure  (a  Gaussian  distribution)  arises  because  the  class 
of  continuous  (but  not  necessarily  differentiable)  functions  is  equivalent  to  the  sample  func¬ 
tions  of  a  Brownian  motion,  and  the  Gaussian  weighting  along  it  is  an  approximation  to 
the  diffusion  of  this  measure  in  time  [Doob,  1984].  The  result,  then,  is  a  two-dimensional 
Gaussian  distribution,  and,  since  each  entry  in  the  tangent  field  is  considered  independently, 
the  total  potential  distribution  is  the  pointwise  summation  of  them. 

More  formally,  the  set  of  generators  of  the  potential  field  is  the  set  of  2D-Gaussians 
{Gi  :  i  =  l,...,n}.  For  each  trace  point  *,•  G{  is  obtained  by  the  multiplication  of  two 
1D-Gaussians,  one  in  the  direction  of  the  quantized  tangent  at  this  point,  and  another  one 
(with  smaller  deviation)  in  the  perpendicular  direction.  Thus,  at  a  trace  point  i  =  (x,-,yj), 
with  tangent  0,- 


Gi(x,y) 


-  ^KEc-{fi{x'y)-Zi)2/<r2E)  ^KBe~i9i{x'y)-yi)2/a b) 


(1) 


where  the  functions  /,(x,y)  and  y,(x,y),  which  rotate  the  x  and  y  axes  to  the  axes  indicated 
by  the  0,  and  0,  +  J  directions  respectively,  are  given  by 


/i(x,  y)  =  n  +  (x  -  x,)  cos  0,  +  (y  -  y;)  sin  0,- 
y»(*,  y)  =  yi  -(x-  Xi)  sin  0,-  +  (y  -  y;)  cos  0,-. 

The  parameter  <rE,  which  is  the  standard  deviation  of  the  lD-Gaussian  in  the  0,-direction, 
controls  how  far  the  local  information  given  by  the  tangent  0j  is  extended  from  the  trace 
point  (xj,yj).  The  parameter  <75,  which  is  the  standard  deviation  of  the  lD-Gaussian  in 
the  (0,  +  ^-direction,  controls  the  width  over  which  the  extended  information  is  valid.  For 
these  reasons,  we  shall  refer  to  the  parameters  crE  and  <73  as  the  extension  parameter  and 
the  blurring  parameter ,  respectively,  and  K E  and  KB  are  (positive)  weights  associated  with 
each  of  the  lD-Gaussians.  Figure  3  presents  a  plot  of  the  resulting  2D-Gaussian. 

The  (tangent  field)  potential  distribution ,  Ujp,  is  then  the  pointwise  summation  of  the 

Gi¬ 

rt 

^TF(xry)  =  Yl 
1=1 
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It  should  be  noted  that  each  generator  G,\  as  defined,  is  a  function  of  the  tangent  estimate 
alone,  with  the  curvature  estimate  negligible.  The  curvature  could  be  used  if  curved,  2D 
Gaussians  G[  were  defined  for  each  tangent  field  entry  i.  G't  would  then  be  a  Gaussian 
along  a  curved  axis  of  curvature  k,  in  the  0,  direction,  multiplied  by  another  Gaussian 
perpendicular  to  the  curved  axis.  However,  the  potential  distribution  Vjp  obtained  by 
pointwise  summation  of  these  G't.  which  is 


n 

Uj'p{x,y)  =  y)  , 

»=1 

would  be  very  similar  to  Ujp  for  the  following  reasons.  First,  the  curvature  is  only  retrieved 
in  a  very  coarsely  quantized  form  by  the  first  stage  of  the  curve  detection  process,  so  the 
generators  G't  would  be  only  weakly  tuned  to  curvature.  Second,  extending  and  summing 
the  tangent  information  implicitly  gives  curvature,  which  is  a  relation  between  neighboring 
tangents.  Thus,  theoretically  there  appears  to  be  no  need  to  use  curved  Gaussians,  and 
experimentally,  potential  distributions  computed  with  straight  Gaussians  yielded  smooth 
curved  valleys. .  Figure  4  shows  examples  of  potential  distributions  and  of  these  valleys, 
where  one  can  see  that  the  result  is  a  smooth  “landscape*,  in  which  the  jaggies  due  to 
sampling,  noise  and  quantization  have  been  removed  *. 

Recall  from  the  introduction  that  the  key  feature  of  the  potential  distribution  is  the 
location  of  the  valleys,  since  it  is  to  the  valleys  that  the  covering  elements  migrate.  Thus 
we  can  now  state  the  claims  behind  this  paper:  (i)  the  valleys  in  the  potential  distribution 
correspond  to  smooth  curves,  and  ( ii )  the  set  of  quantized  tangents  of  these  smooth  curves 
is  precisely  the  tangent  field  used  to  construct  the  distribution.  Moreover,  over  all  the  pos¬ 
sible  sets  of  curves  which  would  project  onto  this  tangent  field,  the  valleys  of  the  potential 
distribution  give  the  most  probable  ones,  in  the  sense  of  Wiener  measure. 

The  rest  of  this  Section  is  devoted  to  an  analysis  of  the  potential  distribution  character¬ 
istics,  in  order  to  prove  the  above  claims.  This  analysis  also  results  in  a  design  criterion  for 
the  blurring  parameter,  which  insures  that  separate  curves  remain  separate. 

The  analysis  is  conducted  for  C1  (i.e.  continuous  with  continuous  first  derivatives)  and 
non-intersecting  curves,  however  some  of  the  tangent  fields  used  in  the  experiments  contain 
intersecting  curves  and  corners  (in  particular,  see  Figure  8).  Other  research  is  currently 
in  progress  to  deal  with  intersecting  curves  and  tangent  discontinuities,  by  extending  the 
potential  distribution  to  a  branched  potential  distribution;  it  will  be  reported  in  a  separate 
paper  2. 


1  Further  computational  experiments  in  which  curved  (G1)  generators  were  used  led  to  final  results  vir¬ 
tually  indistinguishable  from  those  shown  in  this  paper. 

2  Even  if  no  specific  treatment  is  performed  at  intersecting  points  and  tangent  discontinuities,  the  present 
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3  The  potential  distribution 


3.2  Valleys  in  the  potential  distribution 


YVe  now  introduce  the  concept  of  a  valley,  the  locus  of  points  toward  which  the  elements 
of  our  covering  migrate.  To  begin,  given  a  direction  v.  making  an  angle  <p  with  the  positive  x- 
axis.  we  define  a  o-valley  of  the  potential  distribution  Vj'f[x,y)  to  be  a  connected  component 
of  the  set 

{(x,y) :  v^.*)  uTF(x,y)  >  o}, 

where  \7$f{x.y)  and  ^gf{x,y)  denote  respectively  the  first  and  second  directional  derivative 
of  /  in  the  direction  given  by  the  angle  0.  More  generally,  a  valley  of  the  potential  distribution 
UTf(x^)  is  a  connected  subset  of  the  set 

{(*-y) :  V(<t>+%)UTF(x,y)  =  o.  v^+*)  UrF(x,y)  >  0} 

where  <t>  =  <t>(x,y),  i.e.  the  angle  <f>  can  vary  with  the  coordinates  x  and  y.  A  <A-valley  is  then 
a  “straight”  valley  in  the  direction  <p. 

The  idea  behind  the  concept  of  valleys  is  that  of  “directional  minima”.  While  local 
minima  are  points  of  a  valley,  points  of  a  valley  are  not  (necessarily)  local  minima  of  the 
potential  distribution.  The  condition  for  a  point  (x,y)  to  be  a  local  minimum  of  (Jtf  >s  that 
VoUtf(x 1  y)  =  0  and  y)  >  0  for  all  directions  0.  The  definition  of  a  valley  is  thus 

weaker  than  the  definition  of  a  local  minimum,  and  points  in  the  valleys  are  minima  with 
regard  to  a  single  direction,  namely  the  direction  given  at  each  point  by  the  angle  <j>(x,y). 

3.3  Analysis  of  potential  distributions 


The  analysis  of  the  two  simplest  potential  distributions,  namely  a  single  Gaussian,  or 
perfectly  aligned  Gaussians,  is  simple  and  shows  that  the  valleys  are  ^-valleys  (where  <j>  is 
the  direction  of  the  Gaussian)  corresponding  to  the  most  probable  smooth  curve  indicated 
by  the  tangent  field.  The  formal  treatment  of  those  cases  can  be  found  in  [David  and  Zucker, 
1989]. 

In  order  to  achieve  this  result  for  a  more  general  potent i ail  distribution,  we  consider  the 
case  of  two  parallel  Gaussians  (this  also  leads  to  a  criterion  on  the  blurring  parameter  of  the 
Gaussian  weighting  functions  which  insures  curve  separation). 

Observe  that,  when  the  potential  distribution  is  generated  by  several  Gaussians  that 
are  in  general  position,  we  require  the  potential  distribution  to  be  such  that:  (1)  Nearby 
tangent  field  entries  which  lie  along  the  trace  of  a  unique  curve  must  form  a  unique  valley, 
corresponding  to  the  most  probable  position  of  the  curve,  and  (2)  Nearby  tangent  field 

of  the  fingerprint  image  is  correctly  retrieved  (see  Figure  8).  But  further  analysis  would  have  to  be  done 

in  order  to  insure  the  validity  of  the  results  in  the  general  case. 
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entries  belonging  to  two  nearby  curves  must  not  be  blurred  together,  and  must  give  rise  to 
two  separate  valleys. 

These  two  requirements  are  sufficient  to  insure  curve  separation,  since  the  curves  are 
found  by  a  migration  of  covering  elements,  born  at  tangent  field  entry  positions,  over  the 
potential  distribution  (see  Section  2.2).  Then,  in  the  case  of  tangent  field  entries  belonging 
to  different  curves,  the  migration  will  stop  as  soon  as  the  covering  elements  each  drop  into 
their  surrounding  valley.  Hence,  possible  additional  valleys  between  the  two  positions  will 
never  be  found  if  two  valleys  exist  sufficiently  close  to  each  one  of  the  original  positions. 


We  now  concentrate  on  the  second  observation  above,  namely  that  nearly  curves  give 
rise  to  distict  valleys.  This  leads  to  our  design  criterion  for  <73.  Again  consider  only  two 
interacting  Gaussii  ,s,  but  from  a  tangent  field  consisting  of  two  parallel  and  vertical  (i.e. 
0l  =  02  =  £)  tangents,  with  identical  y-coordinates.  Then,  I'pp  =  Gi(x,y)  +  (?2(x,y),  and 
using  9\-  6 2  =  f  i  2/1  =  2/2  in  Eq.  1,  we  get 

Gi(x,y)  =  -A'EA'Be'(<!'-!'l)2/4)£-(l'-Il)2/4) 

(?2(x, y)  =  -I<EKBe'({y~yi)2/<Tlh~({x~X2)2/<TB)  (  ^ 

(see  Fig.  5). 

We  now  look  for  the  ^-valleys  of  Upp  in  this  particular  case.  From  now  on,  and  without 
lost  of  generality,  we  assume  xj  <  X2-  The  critical  points  (for  the  5-valleys)  are  those 
where  SJrUpp  =  0,  or  equivalently,  those  where  -ffe Fpp(x,y )  =  0.  Computing  the  partial 
derivative,  we  obtain 

2  2 

2KEKBe~{y~yi)  /<rE  f  —ir—r,  1„2  .  _^_^2/„2 


'B 


(x  —  xi)e  -f  (x  —  X2)e 


=  0, 


or,  since  first  term  of  the  product  is  always  strictly  positive, 

/  \  -(*-*!  )2/0n  ,  /  \  -(r-Xo)2/<r£  n 

(x  —  x\)e  v  '  B  +  (x  —  X2)e  1  B  =  0. 


(3) 


Then,  by  positivity  of  the  exponential  function,  any  solution  x  of  Equation  3  is  in  the 
ope  i  interval  (xj,X2).  Also,  Equation  3  depends  only  on  Ax  =  X2  —  x\  and  aB.  Without 
lost  of  generality,  set  x\  =  0;  then  X2  =  Ax  and  F{x\,xi,oB)  becomes 

+  (x  -  Ax)e-<l-A')2/,,B  (4) 

which  depends  only  of  Az  and  <rg,  and  which  we  denote  F(x)  for  short. 


Observe  that  the  points  (x,  y)  belonging  to  the  j- valleys  of  (Jpp  are  entirely  characterized 
by  the  function  F(x)  l.  More  precisely,  it  can  be  shown  that  (xo,yo)  belongs  to  a  ^-valley 
of  Upp  if  and  only  if  F(xq)  =  0  and  ■§^F{x o)  >  0. 


'  Note  that  this  also  means  that  the  presence  of  the  ^-valleys  does  not  depend  on  the  y  coordinate. 


3  The  potential  distribution 


Figure  5  (a)  The  tangent  field,  consisting  of  two  entries  with  0j  =  $2  =  ^  and 
VI  =  V2  (b)  .  potential  distribution  obtained  from  (a)  choosing  erg  =  2.75, 
ffg  =  =  0.6875,  Ag  =  Ag  =  1.  This  Gaussian  separates  the  tangents  into  two 

different  valleys,  (c)  A  potential  distribution  obtained  from  (a)  choosing  <rg  =  2.75, 

<rg  =  r¥  «  1.83,  Kg  =  Kg  =  1.  This  Gaussian  blurs  the  tangents  into  the  same 
valley. 


3.  The  potential  distributioc 


A  condition  to  separate  cases  (b)  and  (c)  of  Figure  5  relates  the  distance  Ax  between  tbe 
two  Gaussian  operators  and  the  blurring  parameter  <75.  More  precisely,  it  can  be  deduced 
from  last  result  [David  and  Zucker.  1989]  that  the  vertical  straight  line  x  =  ^  is  a  j- valley 
of  L’tf  an<f  only  A x 2  <  2<jg. 

To  summarize,  the  key  structural  properties  of  the  potential  distribution  are: 


(i)  When  Ax2  <  2<r|,  the  valley  at  x  =  4r  is  unique. 

(ii)  When  Ax2  >  2<7g  so  that  x  =  becomes  a  maximum,  two  valleys  are  created,  one 
at  x  =  Xj  «  xj,  and  one  at  x  =  x£  «  12 ■  Also,  the  valley  at  x  =  is  lost.  The 
slight  variation  in  position  from  x\  and  X2  arises  because  of  a  weak  interaction  between 
nearby  Gaussians;  see  David  and  Zucker  [1989]. 

(iii)  When  Ax  becomes  large,  the  distances  (xj  —  xj)  and  (12  —  x£)  become  small. 

(iv)  When  <79  becomes  small,  the  distances  (xj  —  xj)  and  (x2  —  x£)  become  small. 


The  preceding  analysis  thus  indicates  that  the  valleys  in  the  potential  distribution  have 
precisely  the  properties  required  for  synthesizing  global  curves  from  the  tangent  field.  Before 
proceeding  to  the  study  of  dynamic  curves  for  locating  these  valleys,  we  extend  one  aspect 
of  the  preceding  analysis  into  a  constraint  on  the  parameters  controlling  the  Gaussian  gen¬ 
erators. 

3.4  Design  criterion:  Blurring  parameter 


Continuing  the  particular  case  of  preceeding  Section,  we  know  that  two  (vertical  and 
parallel)  tangent  field  entries  give  rise  to  a  unique  valley  when  Ax2  <  2<x|,  and  to  two 
separate  valleys  when  Ax2  >  2<7g.  Recall  from  the  introduction  that  image  curves  are 
represented  in  the  tangent  field  as  connected  sets  of  tangent  field  entries.  Thus,  when  two 
(parallel  and  vertical)  tangent  field  entries  each  occupy  overlapping  (3x3)  neighborhoods, 
they  belong  to  the  same  curve,  and  when  they  are  separated  by  at  least  one  empty  pixel, 
they  belong  to  different  curves  *.  Then,  any  value  of  the  blurring  parameter  such  that 

l2  <  2a|  <  22  or  -7=  <  og  <  n/2  (5) 

v2 

insures  that  the  two  tangent  field  entries  give  rise  to  separate  valleys  if  and  only  if  they 
belong  to  (locally)  separate  curves. 


1  Or,  they  belong  to  different  parts  of  the  same  global  curve,  i.e.  there  is  a  chain  somewhere  between  tbe 
two  tangents.  But  locally ,  they  must  be  viewed  as  separated. 
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More  generally,  it  is  clear  that  many  other  relations  (in  position  and  orientation),  apart 
from  the  particular  cases  just  studied,  are  possible,  and  lead  to  similar  inequalities.  For 
example,  consider  two  parallel  tangent  field  entries  with  6  —  j.  These  entries  are  then 
located  at  diagonal  pixels  and  this  case  is  just  a  rotation  of  the  preceding  one.  But  now,  the 
distance  between  two  neighboring  tangent  field  entries  is  the  distance  across  the  diagonal, 
i.e.  \/2.  Thus,  any  value  of  <rg  such  that 

v/22  <  2<t|  <  (2V2)2  or  1  <  <73  <  2  (6’ 

insures  separation  of  (locally)  disconnected  sets  of  tangent  field  entries,  and  only  of  those 
sets.  We  submit  that  these  two  cases  are  typical  of  the  general  one,  and  summarize  the 
analysis  by  saying  that  when 


(i)  the  analysis  is  restricted  to  non-intersecting  curves; 


(ii)  curves  of  the  image  are  represented  as  connected  sets  of  tangent  field  entries; 


(iii)  the  choice  of  ctq  is  made  according  to  Equations  5  and  6; 


we  obtain  that 


(1)  Each  connected  component  of  the  tangent  field  will  give  rise  to  a  unique  valley,  built 
from  the  interaction  of  all  the  tangent  field  entries  of  the  component;  thus,  the  valley 
indicates  the  position  of  the  most  natural  curve  which  projects  onto  the  connected 
component  of  tangent  field  entries. 


(2)  Different  connected  components  of  the  tangent  field  will  be  separated  into  distinct 
valleys.  These  valleys  are  not  completely  independent  since  there  is  a  weak  attraction 
between  nearby  valleys  [David  and  Zucker,  1989].  But  this  attraction  is  non-negligible 
only  for  very  close  curves  (a  few  pixels). 


Experimentally,  it  will  be  shown  in  Section  6  that  any  reasonable  value  of  03  relative  to 
Equations  5  and  6  (say  between  0.75  and  1.5)  ensures  that  the  two  above  claims  are  verified. 

4.  Valley  detection 

From  the  tangent  field,  we  derived  a  potential  distribution  in  which  the  valleys  indicate 
the  position  of  the  curves  in  the  image.  We  are  now  ready  to  identify  these  valleys  to  obtain  a 
precise  description  of  the  curves.  Since  the  valleys  are,  by  definition,  connected  loci  of  points 
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that  are  not  all  minima  of  the  potential  distribution,  standard  minimization  techniques  are 
not  adequate.  Rather,  we  use  short  segments  of  curves  to  capture  these  loci,  and  we  define 
a  process  by  which  the  curves  migrate  into  the  valley.  Together  these  segments  will  define 
the  global  covering.  Figuratively,  the  process  works  as  follows.  Suppose  that  we  are  given  a 
rough  approximation  of  where  a  valley  lies  in  the  potential  distribution.  Now,  if  a  curve  is 
placed  at  this  approximate  position,  within  a  potential  well  and  parallel  to  the  valley,  and 
let  slide  as  if  it  were  governed  by  a  kind  of  gravity,  then  it  will  slide  toward  the  bottom  of 
the  well;  i.e..  the  valley.  Moreover,  if  the  curve  is  not  completely  rigid,  it  will  deform  itself 
to  fit  the  exact  shape  of  the  valley.  Then,  at  the  end  of  the  process,  when  “gravitational* 
stability  is  attained,  a  description  of  the  valley  is  given  by  the  associated  curve.  This  is 
precisely  what  we  shall  do. 

Note  that  the  above  mechanism  relies  on  the  fact  that  we  have  a  good  initial  approxi¬ 
mation,  in  position  and  orientation,  of  each  valley.  If  we  place  a  piece  of  curve  randomly  on 
the  potential  distribution  and  let  it  evolve  under  “gravity”,  the  final  position  of  the  curve 
could  be  meaningless  in  terms  of  valley  detection.  More  likely  than  not,  the  curve  would 
span  several  valleys.  Our  initial  approximation  is  given  by  the  tangent  field,  and  the  wells 
by  the  Gaussian  generators.  It  is  analyzed  more  carefully  in  Sec.  5.  In  this  Section,  we 
develop  the  variational  principles  that  govern  the  dynamics  of  each  deformable  curve  indi¬ 
vidually;  the  global  cover  will  then  be  computed  by  running  all  of  these  dynamical  processes 
in  parallel.  For  now,  we  focus  on  one  element  of  the  cover,  and  suppose  it  exists  at  some 
(initial)  position  on  the  potential  distribution.  We  then  develop  a  mathematical  model  to 
govern  the  movement  of  the  curve,  basically  following  that  adopted  for  snakes  ([Kass,  Witkin 
and  Terzopoulos,  1987];  [Terzopoulos,  1987a  and  1987b]),  by  applying  classical  mechanics  to 
deformable  curves. 

A  deformable  curve  v  is  a  differentiable  map  v  ;  R2  — *  R2.  We  write  v(s,t)  = 

( x(s,t),y{s,t ))  with  s  €  SI  the  space  domain,  and  t  €  [fo»*l]  the  time  domain.  We  de- 

note  jfev(s,t)  by  va(s,t),  t)  by  vaa(s,y),  and  $v{s,t)  by  v<(s>y)- 

We  now  define  the  potential  functional  Ul{v)  of  a  curve  v,  where  the  superscript  indicates 
that  the  potential  functional  is  computed  at  each  time  t.  This  corresponds  to  the  potential 
energy  of  the  curve.  It  is  related  to  the  potential  distribution,  but  also  includes  internal 
constraints  on  the  curve  (how  the  curve  is  allowed  to  extend  and  fold).  These  internal 
constraints,  corresponding  to  the  physical  concepts  of  tension  and  rigidity,  must  be  added  in 
order  to  specify  completely  the  movement  of  the  curve.  The  potential  functional  of  a  curve 
v  at  each  time  t  is 


u‘(v)  =  i  J  /(hi2)  +  s(k,|2)  +  UTF(v)d,  (7) 

where  Utf(v)  is  the  value  of  the  potential  distribution  at  the  point  (x(s,<),y(s,<)),  /( |v#|2) 
controls  the  tension  of  the  curve,  and  0(|t>aa|2)  controls  the  rigidity  of  the  curve.  Among  all 


17 


5.  The  global  covering 


the  possible  functions  /  and  g ,  we  simply  choose  the  natural  ones 

/(Ms,*)  |2)  =  u?i(s)Ms,*)|2 
/(M(s.*)|2)  =Ms)M(s.<)!2 

where  uq  and  are  constants  over  time.  A  method  for  choosing  the  constants  is  described 
in  Sec.  5;  we  will  also  see  that  these  simple  forms  are  sufficient  to  control  the  curve  movement 
for  our  purposes. 

Now  that  the  potential  functional  is  completely  defined,  we  can  use  the  methods  of 
classical  mechanics  (principle  of  least  action,  Lagrange- Euler  equations  of  motion)  and  obtain 
a  system  of  differential  equations  which  describe  the  movement  of  the  deformable  curves. 
Then,  with  known  initial  positions,  these  equations  can  be  solved  numerically.  All  details  can 
be  found  in  [Terzopoulos,  1987a],  (Terzopoulos,  1987b]  and  in  the  Appendix  to  this  paper. 

5.  The  global  covering 


We  now  proceed  to  put  all  of  the  previous  pieces  together.  We  infer  the  global  curves 
from  the  tangent  field  by  recovering  a  covering  of  these  curves.  Each  element  of  the  covering 
is  a  deformable  curve  that  moves  according  to  the  model  described  in  Sec.  4,  within  the 
wells  of  the  potential  distribution  described  in  Sec.  3.  Of  course,  we  require  good  initial 
positions  for  the  covering  elements,  so  that,  at  “gravitational”  stability,  the  curves  will  all 
have  moved  into  valleys,  and  we  require  that  each  valley  be  completely  covered  by  curves. 
In  this  Section,  we  precisely  state  what  we  mean  by  a  covering  of  the  global  curves,  and  then 
develop  a  particular  covering  to  meet  the  above  two  requirements. 


5.1  Curves  and  coverings 


As  a  mathematical  object,  a  plane  curve  v  is  a  mapping  from  an  interval  /  C  1R  to  ]R2, 
i.e. 

v(s)  =  (x(a),y(s.))  for  s  €  /. 

The  set  v(I)  C  1R2  is  called  the  trace  of  the  curve  v. 

Since  our  overall  goal  is  applications  in  vision,  we  seek  to  recover  the  trace  of  each  curve 
as  precisely  as  possible.  Once  this  trace  is  available,  it  provides  a  basis  for  inferring  all  the 
other  curve  properties  that  do  not  depend  on  the  exact  form  of  the  mapping.  This  leads 
us  to  consider  equivalence  classes  of  curves  obtained  by  factoring  the  set  of  curves  by  the 
equivalence  relation  given  by  the  equality  of  trace.  More  formally,  we  propose  that  two 
curves  v  :  Iv  — ►  1R2  and  u  :  /„  — »  R2  are  said  to  be  equivalent  if  and  only  if  their  traces 
coincide  i.e.  v(Iv)  =  u(/u).  But  an  image  rarely  consists  of  a  unique  curve,  and  we  want 
to  study  general  classes  of  images.  We  thus  say  that  two  sets  of  curves  U  and  V,  where  U 
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consists  of  the  curves  u  :  Iu  — »  1R^  and  V  consists  of  the  curves  v  :  lv  — ►  R^,  are  said  to  be 
equivalent  if  and  only  if 

U  u(/“)  =  U v('^ 

u£L'  v£V 

Given  a  set  of  curves  V,  we  denote  the  equivalence  class  formed  by  all  sets  of  curves 
equivalent  to  V  by  E(V),  and  any  member  U  of  £(V7)  is  called  a  covering  of  the  set  V',  or 
is  said  to  cover  V.  The  elements  of  U  (which  are  curves  u  :  /„  — »  R2)  are  called  covering 
elements. 

5.2  Construction  of  the  global  covering 


We  now  develop  one  covering  which  enjoys  natural  representational  properties.  Recall 
that  reliable  local  estimates  of  the  curve  positions  and  orientations  are  given  in  the  tangent 
field.  Now,  consider  each  tangent  field  entry  as  a  unit-length  straight  curve,  and  let  these  n 
curves  deform  themselves  according  to  the  model  already  described.  We  then  get  the  set  C 
of  these  n  deformed  curves,  where  each  curve  c  €  C  originated  from  a  tangent  field  entry.  In 
this  Section,  we  will  show  that  the  set  C  is  a  covering  (according  to  the  above  definition)  of 
the  global  curves  in  an  image.  Or  equivalently,  since  the  global  curves  are  described  by  the 
valleys  of  the  potential  field,  we  will  show  that  each  small  curve  c  lies  in  a  valley,  and  that 
each  valley  is  completely  covered  by  curves  of  the  set  C.  More  formally,  the  global  covering 
of  the  curves  in  an  image  is  the  set  C  of  the  n  curves 

Cj(x,*i)  =  *  =  !,•••>«  . 

where  at  initial  time  each  c,-(s,  <o)  *s  a  straight  line  identical  to  the  itfl  entry  of  the  tangent 
field,  and  at  final  time  each  c,(x,  t\)  is  a  dynamic  curve  that  has  moved  onto  a  valley  of 
the  potential  distribution.  Starting  from  now,  the  phrase  “covering  elements”  will  always 
refer  to  the  elements  of  the  global  covering.  We  now  complete  their  dynamics. 

5.3  Covering  element  movement 


Recall  from  the  preceeding  Section  that  for  each  covering  element  c(s,  t)  =  (x(s,  t),  y(s ,  t), 
with  s  €  ft,  the  potential  functional  is 

Ul{c)  =  /  u>i|c,|2  +  U2\ctt\2  +  UTF(c)ds 

J  n 

=  /  UTF(c)ds+  f  wllc*|2  +  W2|c«|2ds  (8) 

JnJn 

where  the  first  term  controls  the  migration  of  the  covering  elements  toward  the  valleys,  and 
the  second  term  controls  how  the  covering  elements  are  allowed  to  deform  themselves  during 
the  migration.  We  call  this  latter  term  the  internal  potential ,  and  set  it  so  that  each  covering 
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element  increases  its  length  to  a  prescribed  length.  The  movement  of  the  covering  elements 
is  thus  a  two-fold  process,  consisting  of  a  migration  and  of  a  length  increase  (see  Fig.  2). 
The  details  are  a s  follows. 

The  internal  potential  of  a  covering  element  c(s,t)  =  (x(s,t)\  y(s,t)),  with  s  €  fl  is 

/  +  ^jcaal^ds,  (9) 

J  n 

where  u>\  and  u>2  control  tension  and  rigidity  respectively.  In  fact,  since  the  minimization  of 
(9)  participates  in  the  covering  elements’  movement,  we  observe  that  the  covering  elements 
tend  to  decrease  their  length  for  positive  values  of  ,  increase  their  length  for  negative 
values  of  u>j,  become  straight  for  positive  values  of  and  fold  and  curve  themselves  for 
negative  values  of  u>%. 

Given  a  prescribed  mandatory  length  Lp ,  we  set  u>\  to  be,  at  each  time  t,  u>i(c)  = 
f(Ll{c)  -  Lp),  where  I*(c)  is  the  length  of  the  covering  element  c(s,t)  at  time  t,  i.e. 

L\c)  =  J  | c6{s,t)\ds  =  ^xs{s,t)2  +  ys(s,t)2ds. 

The  function  /  is  then  simply  a  ramp  function  saturating  when  {L1  (c)  —  Lp\  becomes  big. 
It  is  easy  to  see  that  this  choice  achieves  the  desired  result:  a  covering  element  c  will  tend 
to  extend  when  Ll{c)  <  Lp,  and  to  shrink  when  Ll(c)  >  Lp.  It  should  be  noted  that  is 
independent  of  the  parameter  s,  which  makes  the  tension  equally  distributed  over  the  entire 
covering  element  length. 

Although  the  presence  of  a  rigidity  term  is  essential  to  the  process,  its  exact  value  is  not 
a  key  point,  and  it  suffices  to  set  it  empirically  within  an  order  of  magnitude  to  get  good 
results.  It  was  experimentally  verified  that  changing  the  rigidity  parameter  by  a  reasonable 
amount  does  not  change  the  results  significantly  (see  [David  and  Zucker,  1989]).  But  the 
rigidity  parameter  is  essential  to  the  algorithm,  in  order  to  prevent  a  covering  element  from 
folding  on  itself  while  finding  its  optimal  position,  since  the  potential  distribution  and  the 
tension  parameter  do  not  prevent  those  movements.  We  simply  set  the  rigidity  parameter 
u>2  to  a  constant  value,  the  same  for  each  covering  element,  at  all  times,  and  Over  the  whole 
covering  element  length. 

5.4  Properties  of  the  global  covering 


Now  that  our  model  is  completely  defined,  we  can  state  and  justify  the  claims  made  about 
the  global  covering.  The  global  covering  C,  formed  by  the  n  curves  c,-  which  originated  from 
the  n  tangent  field  entries,  and  which  evolve  according  to  Equation  8,  is  such  that 

(1)  Each  curve  lies  in  a  valley  of  the  potential  distribution. 

(2)  Each  valley  of  the  potential  distribution  is  completely  covered  by  curves  of  the  set  C. 
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Claim  (1)  is  met  by  construction,  since  the  potential  distribution  was  built  from  the 
tangent  field,  i.e.  from  the  curves’  initial  positions.  Then,  each  curve  is,  at  initial  time 
within  the  well  surrounding  its  associated  valley,  and  the  dynamics  drive  the  curve  onto  the 
valley. 

The  truth  of  claim  (2)  is  based  upon  three  facts:  global  curves  project  into  the  tangen: 
field  as  connected  sets  of  tangent  field  entries,  the  length  of  each  covering  element  increases 
during  the  dynamics,  and  there  is  no  movement  of  a  covering  element  outside  the  weL 
surrounding  its  valley.  By  the  last  statement,  we  assume  that  all  valleys  have  a  similar 
depth  along  their  length,  or  equivalently,  that  the  density  of  the  tangent  field  entries  is 
about  the  same  along  the  entire  connected  component.  This  is  guaranteed  by  our  restriction 
to  smooth  non-intersecting  curves. 

Note  that  the  diagonal  length  across  a  pixel  (which  is  the  maximal  distance  between  two 
neighboring  elements  of  a  connected  set)  is  \fl  units;  therefore,  to  guarantee  that  neighboring 
covering  elements  overlap  to  give  the  global  curves,  we  just  have  to  let  each  covering  element 
extend  in  length  to  some  value  greater  than  \/2,  since  each  covering  element  stays  around  its 
initial  position.  Typically,  a  prescribed  length  of  3  is  used,  in  order  to  guarantee  the  overlap. 

Since  the  valleys  of  the  potential  field  describe  the  global  curves  of  the  image,  we  also 
get  that  the  global  covering  is  a  covering  of  the  curves  in  an  image,  i.e. 

ljc(fUi)=  U  •></„), 

c€C  v€V 

where  V  is  the  set  of  the  global  curves  of  the  image. 

6.  Results 


We  now  illustrate  the  computation  of  the  global  covering,  beginning  with  artificial  images 
to  check  the  validity  of  the  algorithm  against  known  examples,  and  finally  with  natural 
images  to  demonstrate  its  robustness. 

The  first  test  shows  that  simple  analytic  curves  are  retrieved  quite  exactly  from  the  tan¬ 
gent  field  information,  i.e.  that  the  “natural”  curves  indicated  by  the  potential  distribution 
correspond  to  the  expected  ones.  The  image  consists  of  two  nearby  circles,  with  different 
radii,  projected  over  a  discrete  sampling  grid  (Fig.  6(a)). 

We  simulate  the  first  stage  of  the  process  (see  Sec.2.1),  which  extracts  the  trace  and 
tangent  of  these  curves  in  the  following  manner.  Each  square  of  the  sampling  grid  traversed 
by  one  of  the  circles  belongs  to  the  trace  of  this  circle,  and  the  tangent  at  each  trace  point 
is  computed  assuming  that  the  curve  crosses  the  center  of  the  square;  afterwards,  all  the 
computed  tangents  are  quantized  into  eight  classes  (Fig.  6(b)).  We  then  run  our  algorithm 
on  the  simulated  tangent  field.  This  tangent  field  is  the  only  information  that  the  second 
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Figure  6  (a)  Two  nearby  circles  of  different  radaii  lie  over  a  discrete  sampling  grid 
(25  x  25).  Their  projection  onto  the  grid  is  given  by  those  pixels  traversed  by  one  of 
the  two  circles,  (b)  The  tangent  field  for  the  two  circles,  consisting  of  the  discretised 
tangent  at  each  trace  point.  Note  that  the  tangents’  positions  also  correspond  to 
the  covering  elements  initial  positions,  (c)  From  their  initial  positions  shown  in  (b), 
the  covering  elements  have  migrated  into  the  valleys  of  the  potential  distribution, 
where  they  attain  stability.  The  parameters  of  this  experiment  are:  og  =  2.75, 
°B  ~  ~  0-58,  Lp  =  3.0;  wj  is  determined  by  R  =  1.0,  K  =  1.0,  and  u>2  =  1.0; 

p  =  2.0, 7  =  2.0.  (d)  The  final  covering  elements’  positions,  superimposed  over  the 
initial  curves. 
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stage  of  the  process  uses  to  retrieve  the  global  curves,  the  initial  circles  being  lost  by  the 
projection  onto  the  sampling  grid.  The  result  is  shown  in  Fig.  6(c)  and  (d).  The  curves 
retrieved  from  the  potential  distribution  correspond  very  accurately  to  the  initial  circles. 
Also,  the  value  of  erg.  chosen  according  to  the  design  criterion,  allows  a  clear  separation 
between  the  two  nearby  circles,  separated  by  only  two  pixels  on  the  sampling  grid. 

Finally,  we  run  our  algorithm  on  three  natural  images,  a  fingerprint  image,  a  radiograph 
of  blood  vessels  in  the  brain,  and  a  satellite  image  of  logging  roads.  Since  we  are  interested 
in  precise  extraction  of  the  curves,  without  blurring  of  very  close  curves,  we  choose  to 
experiment  with  small  images  so  that  details  could  be  examined  up  close.  Figure  7  presents 
the  original  images,  and  the  resulting  global  curves.  An  exploded  display  of  the  first  two 
tests  is  then  presented  in  the  last  two  figures. 


7.  Summary  and  Conclusions 


In  this  paper  we  developed  an  algorithm  for  finding  a  representation  of  the  global  curves 
in  an  image.  The  novel  feature  of  this  representation  is  that  it  is  formulated  in  mathematical 
terms  as  a  covering,  so  that,  rather  than  computing  global  curves  directly,  short  segments 
of  curve  are  computed  independently  from  one  another.  The  short  segments  thus  become 
the  elements  of  the  cover,  and  global  curves  are  given  as  their  union.  This  enables  us  to 
define  the  algorithm  in  a  parallel  fashion,  and  to  avoid  the  a  priori  specification  of  global 
parameters  (e.g.,  the  length  of  the  curve). 

Since  the  global  representation  is  built  up  from  local  ones,  the  algorithm  for  comput¬ 
ing  global  curves  via  coverings  requires  reliable  local  information  about  the  curve.  Within 
our  context,  which  is  a  large  project  on  curve  detection  incorporating  both  mathematical 
and  biological  constraints,  this  local  information  is  provided  by  a  tangent  field,  or  a  list  of 
(quantized  trace)  points  through  which  the  curve  passes,  together  with  coarse  estimates  of 
its  tangent  and  curvature  at  those  points.  The  key  point  to  emphasize  here  is  that  these  local 
properties  need  not  be  specified  to  high  precision;  the  interpolation  properties  implicit  within 
the  covering  algorithm  can  deal  successfully  with  coarse  initial  information.  Thus  the  algo¬ 
rithm  fits  into  a  very  natural  information  processing  hierarchy,  with  both  degree-of-precision 
and  local-to-global  axes  spanned. 

The  local  (tangent  field)  information  is  used  in  two  different  ways.  First,  a  global  de¬ 
scription,  in  the  form  of  a  potential  distribution,  is  synthesized  from  it.  By  definition,  valleys 
in  this  potential  distribution  correspond  precisely  to  curve  locations.  These  valleys  are  de¬ 
tected  by  a  dynamical  process  in  which  the  covering  elements  are  driven  by  the  potential  to 
both  migrate  in  position  and  elongate  in  length.  The  second  use  of  the  local  (tangent  field) 
information  is  in  specifying  the  initial  positions  for  each  covering  element.  Convergence  is 
thereby  guaranteed  by  construction.  Equivalence  between  the  curves  in  the  image  and  the 
covering  elements  is  demonstrated  both  theoretically  and  by  example. 
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Figure  7  Global  curves  of  natural  images.  Top:  A  fingerprint  image.  Middle:  A 
radiograph  of  blood  vessels  in  the  brain.  Bottom:  A  satellite  image  of  logging  roads. 
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Figure  8  Illustration  of  the  first  experiment  on  a  fingerprint  image,  (a)  The  initial 
covering  elements,  at  tangent  field  entries’  positions,  (b)  Final  positions  of  the 
covering  elements;  clearly  the  covering  elements  have  migrated  into  the  valleys,  and 
have  overlapped  to  form  a  covering.  Also  note  that  the  bifurcation  in  the  center  of 
the  image  have  been  recovered  correctly,  without  special  treatment. 


25 


Summary  and  Conclusions 


Figure  9  The  second  test  is  performed  on  a  biomedical  image,  a  cerebral  angiogram, 
(a)  The  initial  positions  of  the  covering  elements,  corresponding  to  the  tangent  field 
entries.  This  image  is  very  noisy,  and  low  contrast.  The  tangent  field  was  computed 
under  the  assumption  that  all  vessels  were  of  comparable  contrast,  (b)  The  final 
positions  of  the  covering  elements.  The  region  where  nearby  covering  elements  do 
not  overlap  completely  corresponds  to  a  region  where  the  tangent  field  was  not 
completely  connected. 
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Since  the  global  curves  are  represented  by  a  covering  that  evolves  according  to  a  dynamic 
process,  we  refer  to  them  as  dynamic  global  coverings.  Although  we  concentrated  on  smooth, 
non-intersecting  curves  in  this  paper,  additional  research  is  in  progress  to  guarantee  the 
validity  of  the  results,  in  the  general  case.  It  works  a s  follow’s.  Since  the  tangent  field 
indicates  curve  intersections  and  discontinuities  by  multiple  tangent  field  entries  at  the  same 
grid  coordinate,  the  potential  distribution  is  split  into  several  “layers”,  one  per  tangent  field 
entry,  in  the  neighborhood  of  each  of  those  coordinates.  Take,  for  example,  the  case  of  two 
curves  intersecting  at  a  single  point.  The  tangent  field  entries  of  those  curves  will  give  rise  to 
two  independent  layers  in  the  potential  distribution  at  this  point,  and  the  process  described 
in  this  paper  can  then  be  applied  at  each  layer.  (The  advantage  of  separating  the  tangent 
field  entries  is,  of  course,  to  eliminate  inappropriate  interactions  between  the  separate  curve 
segments.) 

The  remaining  step  from  a  covering  to  a  global  representation  is  a  specification  of  which 
covering  elements  belong  to  the  same  global  curve.  (It  is  usually  the  case,  for  example,  that 
there  are  multiple  curves  within  a  single  image.)  We  accomplish  this  by  a  straightforward 
exploitation  of  topological  connectivity:  graphically,  the  inital  covering  elements  are  all 
born  with  a  different  “color”;  as  two  covering  elements  overlap  dynamically,  they  become 
the  same  “color”.  In  the  end,  “color”  propagates  along  connected  components,  and  the 
elements  belonging  to  the  covering  of  the  same  curve  are  all  the  same  color,  while  the  covers 
corresponding  to  different  curves  are  different  colors. 

In  conclusion,  we  believe  that  dynamic  coverings  have  application  to  a  variety  of  problems 
that  must  face  the  transition  from  local  to  global  representations  in  a  parallel,  efficent  way. 
For  example,  an  extension  to  surface  coverings  could  work  as  follows.  We  know  that  it  is 
possible  to  extract,  from  3D  data,  an  intermediate  structure  that  is  the  3D  equivalent  of  the 
tangent  field  (see  Sander  and  Zucker  (1989]).  This  structure  mainly  consists  of  an  estimate 
of  the  tangent  plane  at  each  (3D)  trace  point.  Starting  from  these  planes,  a  generalization 
of  the  model  presented  in  this  paper  would  evolve  those  local  estimates  to  fit  together  into  a 
(smooth)  covering  of  the  surface.  The  covering  elements  would  thus  be  overlapping  surface 
patches. 
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APPENDIX 


A.i 


Dynamics  of  the  deformable  curve 


Given  the  potential  functional  Ut( v ),  we  can  define  the  kinetic  energy  of  the  curve. 
Following  classical  mechanics,  the  kinetic  energy  functional  at  each  time  t  is 

Tt(V)~\JQfi\Vt^ds' 

where  p  is  the  (constant)  mass  density.  * 

According  to  the  principle  of  least  action ,  the  motion  of  the  deformable  curve  during  the 
time  interval  (to>*l)  >s  described  by  the  functions  x(s,t),  y(s,t)  for  which  the  integral 
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of  the  Lagrangian  lS{v)  =  —  £/*(u)  is  a  minimum.  It  is  also  known  that  any  extrema  of 

Ll(v)dt  must  satisfy  the  Euler- Lagrange  equations.  Setting  f(v)  =  £(^|t>*|2  -u>i|t>e|2  - 
u>2|vsa|2  -  1'tf(v))i  the  Euler- Lagrange  equations  are 

d  d  a2  a2  52 
■k  ”  a7(^  ~  Tt +  +  +  =  0 

d  d  52  52  52  '  ■  1 1 

~  ds^ys ^  ~  dt^m * +  ds*^y“ ^ +  J  +  a^^y«< ^ =  0 

where,  for  example,  fXgt  denotes  the  derivative  with  respect  to  the  variable  Xgt- 

Equations  .1  hold  for  a  conservative  system.  In  order  to  dissipate  the  kinetic  energy 
produced  during  the  motion,  an  energy  dissipation  functional  can  be  introduced,  such  that 
the  deformable  curves  reach  stable  equilibrium  positions.  Using  the  Raleigh  dissipation 
functional  Dt(v)  =  /fi7|v*|2ds,  where  7  is  the  (constant)  damping  density,  (.1)  becomes 

~  ds^Zs^  ~  di^z^  +  ds*^1**}  +  W^Ztt^  +  dsdi^Zat^ +  dxt^Vt^  =  0 
fy  ~  J$Uvs)  ~ +  £2^)  +  Jjjtfw)  +  =  °- 


Letting  rx(u)  denote  ^Utf(v)>  we  calculate 


fz  -  -fz(v) 


a2 


a2 


a<2  (/*«)=  &2(«1*«) 

a  w = **« 

a2  ,  a2  ,, 

=  asar7**^  _0‘ 


Using  identical  results  for  fy,  £2 (/»«)  •  $(/w)  ’  £z(fytt)  311(1  &(/vrt)’  the 

Lagrange  equations  of  motion  of  the  deformable  curve  are 


d  V  1 

mt  +  7  **  -  +  ^jfo***)  -  ~2r*^^ 

d  ,  x  02,  X  U  #  X 

mt  +  7yt  -  ^(u>iy«)  +  =  -gW)- 


(.2) 
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A  *2  Numerical  solution 

The  solution  of  System  .2  gives  the  positions,  in  space  and  in  time,  of  the  curve  v 
during  the  motion.  We  now  address  the  numerical  approximation  to  System  .2,  following 
Terzopoulos  [1988b]. 

The  space  domain  fi  is  tessellated  into  N+l  nodes  {0,  h ,  2 h. . . . ,  Nh],  h  =  ^r.  Then,  the 
solutions  to  the  system  are  the  vectors  x*  =  (x{ih,t))fLQ  =  (xj)^0,  y 1  =  (y(iM))i=o  = 
(ifi)i-O  at  eac^  t*me  ^  *s  Poss>ble  to  express  (.2)  as  the  linear  system 


Ax *  =  glx,  A?  =  g\. 

Assuming  temporarily  that  the  curve  is  closed,  two  initial  configurations  (initial  conditions) 
are  needed.  Solving  for  x*,  given  a  time  step  A t,  n  and  are  approximated  by 

x*  -  i<-2A< 


*  -  J 


xt-2At  _  2xt-At  + 

*“  "  A(2 

where  backward  differences  are  used,  since  we  can  only  rely  on  curve  positions  at  previous 
times.  This  results  in  the  system  of  N  +  1  equations 

d  ^2 

-  ^(“1**)  +  -pto1")  +  (^2  +  2Af 

_  _Ir  (xt-At  t-Atx  ,  9  _  (Jt _ l_w<-2A*  (-3) 

“  9lr^«  » Vi  >  +  *a*2  xi  9A  i)Xi 


where  is  evaluated  at  time  t  -  At. 

The  right  hand  sides  of  Equations  .3  only  depend  on  prior  configurations,  and  can  be 
evaluated  at  each  time  step  t,  as  long  as  two  initial  configurations  (x*0“Ai  x<q-2A^  m 

given  at  the  initial  time  tQ.  Setting 


J  _  (  (J-At  t 

9z  ~~  l  2^*  Z*  *^» 


-At\  .  2  JLxt-At  _,± _ 1_w(-2aAN 

,  +  2A!2i  Ai2  2A<  '■  )j=0 


and  writing 


(.3)  reduces  to 


(a/  +  K)x*  =  g\ 
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where  the  only  unknown  is  the  stiffness  matrix  K  determined  by  the  relations  between  nodes 
at  each  time  t.  Again  using  finite  differences  to  approximate  the  spatial  derivatives  ^j(u>i xt) 

and  -^(^iss)  (see  Section  .4).  we  get 
d  d ^ 


xi-2  h 


u>2  (i  —  A) 


+  xi-h 
+  xi 


h 4 

— 2a>2(i)  2ufy(i  —  A)  u>i(t  —  A) 


A 4 


A4 


A2 


uyji  +  h)  ,  4u>2(i)  uy(i-h)  vrfi)  -  A) 

I  *  A  •  i  A  I 


+  xi+2h 


h4  '  h4  T  h4  2  A2 
-2w2<i  +  A)  2u^}(t)  wj(i')] 


A2 


h4 

(i  +  A) 


A4 


A2 


A4 


in  the  case  of  closed  curves  (i.e.  x^  =  x^,  x*_2  =  xjy_j,  x^+I  =  ^+2  =  *!)• 

Finally,  setting 


bi  =  - 
a  = 


V2(i  +  A) 
h4 

2u^(i)  _  2uq(i  +  h)  _  wi(») 
h4  h4  ~  A2 

tJ2(i-A)  4w2(»)  ,  u^(t'  +  A)  ui{i-h)  ^i(t) 


h4  h4  h4  A2  A2 

for  t  =  1, . . . ,  N,  the  stiffness  matrix  K  is  the  pentadiagonal  symmetric  matrix 

/  cq  Aq  ao  ajv_!  bN  \ 

6q  ci  6i  ai  an 

°0  Ai  C2  &2  a2 

ai  62  c3  63  03 

an-K  A;v_3  Cff-2  *>N- 2  aN-2 

Otf-1  <*Af-3  AjV-2  cjy_i  dtf_i 

\  AjV  ajy  aAT-2  *AT— 1  CN  ) 

Then,  A  =  0/  +  K  is  also  pentadiagbnal  symmetric  and  can  be  solved  very  efficiently  in 
O(N)  time  and  space  [Benson  and  Evans,  1977],  by  factoring  A  into  triangular  and  diagonal 
matrices. 


.3  Inserting  position  discontinuities 


Up  to  now,  the  deformable  curve  was  assumed  to  be  closed.  We  now  relax  this  assumption 
by  inserting  a  discontinuity  between  node  N  +  1  and  node  0. 
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To  insert  a  position  discontinuity  at  s  =  between  nodes  i  -  1  and  t,  the  stiffness 

matrix  K  has  to  be  changed  to  eliminate  relations  between  nodes  i  —  1  and  i.  The  equations 
which  relate  nodes  i  -  1  and  t  are  given  by  the  matrix  rows  i  —  2,  i  —  1,  i  and  i  +  1,  and  are 
the  following 


a,-_4Zj_4  +  6;_3Xj_3  +  c,_2X,_2  +  bi_2Xi_j  +  aj_2Xi  =  2 

a,_3ij_3  +  6,_2X,_2  +  Ci-i*i-l  +  bj.jXi  +  aj_jxl+i  =  g,_j 

a,_2Xi_2  +  bj_1Xi_1  +  qxj  +  6,xl+j  +  ajX,+2  =  9i 
ai-lxi-I  +  bi*i  +  C«+1I«+1  +  bi+lxi+2  +  ai+lxi+3  =  9i+ 1- 


From  (.4),  we  immediately  get  6,_i  =  a,_2  =  a»- 1  =  0,  which  implies  0^(1  —  1)  = 
u>2(i)  =  u>i(t  —  1)  =  0.  This  dictates  the  9  entries  to  change  in  the  matrix,  which  are 


1.  a,_2  =  0 

2.  a,_i  =  0 


3.  6 i_j  = -  *1^ 


6.  Cj_2 

7.  Cj_! 


W9(i-3A)  .  w9(«'-2/i)  ,  wi(i'-3/»)  ,  wi(»-2A) 

V  +  +  t*-1  +  “V-^ 

u>2(i-2/i)  wi(i-2A) 

+  hi 


8. 

9. 


Ct'+l 


w2i<+2^ 
T  *4 


+ 


+  “!(•+'*) 


In  our  implementation,  this  is  used  only  once,  to  relax  the  condition  that  the  curve  must 
be  a  closed  one. 


A. 4  Development  of  the  spatial  derivatives 


We  indicate  here  how  the  numerical  approximation  to  spatial  derivatives  are  obtained. 
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Centered,  backward  and  forward  differences  are  mixed  in  order  to  obtain  a  symmetric  matrix. 


d  ,  ,  a>i(t)x,(i)  -  u\(i  -  h)x,(i  -  A) 

Fs^'x,)  * - 1 - 


r*«+A  -  Jii  un(j-h) 


r  x*  ~  xi-h 


A 

l  A  J 

A  l  A  J 

u>l(*  -  A)]  , 

-  A)  - 

+  xi+h 

xi—h 

.  A2  J  +  * 

A2 

A2  J 

d2  ,  v  u>2(i  +  h)x8s(i  +  A)  -  2u^{i)xss{i)  +  0^(1  -  h)xS8{i  -  A) 

a?{^x,a) 55 - P - 

_  u>2(i  +  A)  \Xj+2h  ~  ?*,■+/>  +  Xj] 


A 2 
2^(0 
A2 


A2 

*«+A  ~  2x,-  +  Xj_fc 


A2 


tJ2(t  -  A)  r  j,-  -  2x{_h  -I-  x'j-2h} 


=xi~2h 


A2  1 

^2 (i  -  A) 


A2 


A4 


+  ^i-A 


— 2u^2  ( 0  —  2u^(t  —  A)' 


A4 


ru?2(i  +  A)  +  4tJ2(i)  +  u^(i  -  A) 

+Iu - p - 

r-2u>2(i  +  A)-  2^2(1) 

+  i+h  l - J? - 

fW2(i  +  A) 

+  *i+2A 


A4 


Thus,  -^(u;ixa)  +  ^-(u^xjs)  is  numerically  approximated  by 

r^2  (*  “  A) 


-2u^(i)  2u>2 (*  —  A)  u>i(i  -  A)] 


+. 

-[■ 


A4 


A2 


fW2(i  +  A)  .  4w2(t)  .  U2(i-h)  ,  u>i(i)  ,  u>i(i-A)] 
+  'I  A4  +  ~~A4“’  +  A4  +  "2A2~  + 


(» ~  fc)l 
A2  J 


+  xi+h 


— 2u>2(»  +  A)  2u>2(:)  wj  (i  —  A) 


A4 

u>2(*  +  A) 


A4 


A2 


ru>2(*  -1 
+  I«+2A  - £4 
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